Weaviate Podcast

Agentic RAG with Erika Cardenas - Weaviate Podcast #109! Nov 13, 2024

Hey everyone! Thank you so much for watching the 109th episode of the Weaviate Podcast with Erika Cardenas! Erika, in collaboration with Leonie Monigatti, have recently published "What is Agentic RAG". This blog post that was even covered in VentureBeat with additional quotes from Weaviate Co-Founder and CEO Bob van Luijt! This podcast continues the discussion on all things Agentic RAG, covering the basics of Agents, how Agentic RAG changes the game compared to Vanilla RAG systems, Multi-Agent Systems and CrewAI / OpenAI Swarm, Letta, DSPy, and many more! The podcast also anchors by discussing Agentic Generative Feedback Loops and how we are using Agents to improve the quality and expand the capabilities of Generative Feedback Loops!

Let Me Speak Freely? with Zhi Rui Tam - Weaviate Podcast #108! Nov 07, 2024

JSON mode has been one of the biggest enablers for working with Large Language Models! JSON mode is even expanding into Multimodal Foundation models! But how exactly is JSON mode achieved?

There are generally 3 paths to JSON mode: (1) constrained generation (such as Outlines), (2) begging the model for a JSON response in the prompt, and (3) A two stage process of generate-then-format.

I am BEYOND EXCITED to publish the 108th Weaviate Podcast with Zhi Rui Tam, the lead author of Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models!

As the title of the paper suggests, although constrained generation is awesome because of its reliability, we may be sacrificing the performance of the LLM by producing our JSON with this method.

The podcast dives into how these experiments identify this and all sorts of details about the potential and implementation details of Structured Outputs. I particularly love the conversation topic of incredible Complex Structured Outputs, such as generating 10 values in a single inference.

I hope you enjoy the podcast! As always please reach out if you would like to discuss any of these ideas further!

SWE-bench with John Yang and Carlos E. Jimenez - Weaviate Podcast #107! Oct 30, 2024

Hey everyone! Thank you so much for watching the 107th episode of the Weaviate Podcast! This one dives into SWE-bench, SWE-agent, and most recently SWE-bench Multimodal with John Yang from Stanford University and Carlos E. Jimenez from Princeton University! One of the most impactful applications of AI we have seen so far is in programming and software engineering! John, Carlos, and team are at the cutting-edge of developing and benchmarking these systems! I learned so much from the conversation and I really hope you find it interesting and useful as well!

AI in Education with Rose E. Wang - Weaviate Podcast #106! Oct 22, 2024

Hey everyone! I am SUPER excited to publish the 106th episode of the Weaviate Podcast featuring Rose E. Wang!! Rose is a Ph.D. student at Stanford University where she has lead incredible research at the cutting-edge of AI applications in Education. The podcast heavily discusses her recent work on Tutor CoPilot! Tutor CoPilot is one of the world's largest randomized control trials on the impact AI is having on education, testing 900 students and 1800 tutors in grades K-12. I think this is such an inspiring study and it is interesting to see the data coming in quantifying the impact AI is having on education. I was amazed by the depth of how Rose things about education and learning strategies and how well she integrates cutting-edge topics in AI! I hope you find the podcast interesting and useful!

Compound AI Systems with Philip Kiely - Weaviate Podcast #105! Oct 17, 2024

Hey everyone! Thank you so much for tuning into the 105th episode of the Weaviate Podcast! This one features Philip Kiely diving into all sorts of apsects related to Compound AI Systems! We are now seeing far better results with AI models by breaking up tasks into multiple stages and inferences. Philip explains the work they are doing at Baseten to optimize and scale deployments of these emerging systems and all sorts of aspects about them from Structured Generation to their distinction with Agents! I hope you find it useful!

AI Agents That Matter with Sayash Kapoor and Benedikt Stroebl - Weaviate Podcast #104! Sep 18, 2024

AI Researchers have overfit to maximizing state-of-the-art accuracy at the expense of the cost to run these AI systems! We need to account for cost during optimization. Even if a chatbot can produce an amazing answer, it isn't that valuable if it costs, say $5 per response! I am beyond excited to present the 104th Weaviate Podcast with Sayash Kapoor and Benedikt Stroebl from Princeton Language and Intelligence! Sayash and Benedikt are co-first authors of "AI Agents That Matter"! This is one of my favorite papers I've studied recently which introduces Pareto Optimal optimization to DSPy and really tames the chaos of Agent benchmarking! This was such a fun conversation! I am beyond grateful to have met them both and to feature their research on the Weaviate Podcast! I hope you find it interesting and useful!

MIPRO and DSPy with Krista Opsahl-Ong! - Weaviate Podcast #103 Aug 28, 2024

I am beyond excited to publish our interview with Krista Opsahl-Ong from Stanford University! Krista is the lead author of MIPRO, short for Multi-prompt Instruction Proposal Optimizer, and one of the leading developers and scientists behind DSPy!

This was such a fun discussion beginning with the motivation of Automated Prompt Engineering, Multi-Layer Language Programs (also commonly referred to as Compound AI Systems), and their intersection. We then dove into the details of how MIPRO achieves this and miscellaneous topics in AI from Structured Outputs to Agents, DSPy for Code Generation, and more!

I really hope you enjoy the podcast! As always, more than happy to answer any questions or discuss any ideas about the content in the podcast!

AI-Native Development with Guy Podjarny and Bob van Luijt - Weaviate Podcast #102! Aug 14, 2024

AI is completely transforming how we build software! But how exactly? What does it mean for a software application to be AI-Native versus AI-Enabled? How many other aspects of software development and creativity are impacted by AI?

I am super excited to publish our 102nd Weaviate Podcast with Guy Podjarny and Bob van Luijt on AI-Native Development!

Guy Podjarny is a co-founder of Snyk, a remarkably successful Cybersecurity company. He is now back on the founder journey, diving into AI-Native Development with Tessl!

Guy and Bob both have so much expertise in how software is developed and shipped to the world. There are so many interesting nuggets in this from defining AI-Native to Stateful AI, AI-assisted coding, subjectivity in software specification, personalized content, and much more!

I hope you enjoy the podcast, this was a really fun and interesting one!

Scaling Pandas with Devin Petersohn - Weaviate Podcast #101! Jul 17, 2024

Hey everyone! Thank you so much for watching the 101st episode of the Weaviate Podcast with Devin Petersohn! Devin is the creator of Modin, one of the world's most advanced systems for scaling Pandas! Devin then went onto co-found Ponder, which was acquired by Snowflake in early 2023. This was one of my favorite podcasts of all time, I learned so much about the internals of Data Systems and I hope you do as well!

Generative UIs with Lucas Negritto and Bob van Luijt! Jul 04, 2024

What is an AI-native application? This has been one of the questions we are most interested in answering at Weaviate! This podcast explores this question with Weaviate Co-founder Bob van Luijt and Lucas Negritto. Formerly at OpenAI, Lucas is now building Odapt, a remarkable example of such an application where we no longer use front-end code, rather rendering the UI entirely within the generative model!! There are many interesting topics covered such as of course, firstly how this works and how you build these systems, as well as native multimodality, subjective feedback, and more! I hope you find the podcast interesting and useful!

ACORN with Liana Patel and Abdel Rodriguez - Weaviate Podcast #99! Jun 25, 2024

Liana Patel is a Ph.D. student at Stanford University who is the lead author of ACORN, a breakthrough in Approximate Nearest Neighbor Search with Filters! Also joining the podcast is Abdel Rodriguez, a Vector Index Researcher and Engineer at Weaviate. This podcast dives into all sorts of details behind ACORN. Starting with how Liana developed her interest in Approximate Nearest Neighbor Search algorithms and then transitioning into how ACORN differs from previous approaches, the Two-Hop Neighborhood Heuristic, Predicate Subgraphs, Experimental Details, and many more topics! Major thank you to Liana and Abdel for joining the podcast, this was such a fun conversation packed with insights about Proximity Graph algorithms for Vector Search with Filtering!

Window Search Tree with Josh Engels - Weaviate Podcast #98! Jun 19, 2024

Josh Engels is a Ph.D. student at MIT who has published several works advancing the state of the art in Vector Search. Josh has recently developed the Window Search Tree, a new algorithm particularly targeted for improving Filtered Vector Search. Even more particularly than that, the WST algorithm targets Filtered Search with continuous-valued filters such as "price" or "date", also known as range filters. This is a huge application for Vector Databases and it was incredible getting to pick Josh's brain on how this works and the state of Approximate Nearest Neighbor Search!

The Future of Search with Nils Reimers and Erika Cardenas - Weaviate Podcast #97! Jun 11, 2024

Hey everyone! I am SUPER excited to publish our 97th Weaviate Podcast on the state of AI-powered Search technology featuring Nils Reimers and Erika Cardenas! Erika and I have been super excited about Cohere's latest works to advance RAG and Search and it was amazing getting to pick Nils' brain about all these topics!

We began with the development of Compass! Nils explains the current problem with embeddings as a soup!! For example, imagine embedding this video description, the first part is about the launch of a podcast, whereas this part is about an embedding algorithm -- how do we form representations of multi-aspect chunks of text?

We dove into all the details of this from the distinction of multi-aspect embeddings with LLM or "smart" chunkers, ColBERT, "Embed Small, Retrieve Big", and many other topics as well from Cross Encoder Re-rankers to Data Cleaning with Generative Feedback Loops, RAG Evaluation, Vector Quantization, and more!

I really hope you enjoy the podcast! It was such an educational experience for Erika and I and we really hope you enjoy it as well!

Deep Learning with Letitia Parcalabescu - Weaviate Podcast #96! Jun 05, 2024

Hey everyone! Thank you so much for watching the 96th episode of the Weaviate podcast featuring Letitia Parcalabescu! While completing her Ph.D. studies at the University of Heidelberg, Letitia started her YouTube channel: AI Coffee Break with Letitia! Her videos break down complex concepts in AI with a creative mix of technical expertise and visualizations unlike anyone else in the space!We began the podcast by discussing our shared background in creating content on YouTube from starting, to plans for the future, and everything else in between!We then discussed the evolution of Deep Learning over the last few years -- from neural network architectures to datasets, tasks, learning algorithms, and more! I think particularly we are at a really interesting time in the future of learning algorithms! We discussed DSPy and new ways of thinking about instruction tuning, example production, gradient descent, and the future of SFT vs. DPO-style techniques!

Google Cloud Marketplace with Dai Vu and Bob van Luijt - Weaviate Podcast #95! May 07, 2024

Hey everyone, thank you so much for watching the 95th Weaviate Podcast! We are beyond honored to feature Dai Vu from Google on this one, alongside Weaviate Co-Founder Bob van Luijt! This podcast dives into all things Google Cloud Marketplace and the state of AI. Beginning with the proliferation of Open-Source models and how Dai sees the evolving landscape with respect to things like Gemini Pro 1.5, Gemini Nano and Gemma, as well as the integration of 3rd party model providers such as Llama 3 on Google Cloud platforms such as Vertex AI. Bob and Dai continue to unpack the next move for open-source infrastructure providers and perspectives around "AI-Native" applications, trends in data gravity, perspectives on benchmarking, and Dai's "aha" moment in AI!

ParlayANN with Magdalen Dobson Manohar Apr 24, 2024

As you are graduating from ideas to engineering, one of the key concepts to be aware of is Parallel Computing and Concurrency. I am SUPER excited to share our 94th Weaviate podcast with Magdalen Dobson Manohar! Magdalen is one of the most impressive scientists I have ever met, having completed her undergraduate studies at MIT before joining Carnegie Mellon University to study Approximate Nearest Neighbor Search and develop ParlayANN. ParlayANN is one of the most enlightening works I have come across that studies how to build ANN indexes in parallel without the use of locking.

In my opinion, this is the most insightful podcast we have ever produced into Vector Search, the core technology behind Vector Databases. The podcast begins with Magdalen’s journey into ANN science, the issue of Lock Contention in HNSW, further detailing HNSW vs. DiskANN vs. HCNNG and pyNNDescent, ParlayIVF, how Parallel Index Construction is achieved, conclusions from experimentation, Filtered Vector Search, Out of Distribution Vector Search, and exciting directions for the future!

I also want to give a huge thanks to Etienne Dilocker, John Trengrove, Abdel Rodriguez, Asdine El Hrychy, and Zain Hasan. There is no way I would be able to keep up with conversations like this without their leadership and collaboration.

I hope you find the podcast interesting and useful!

RAGKit with Kyle Davis - Weaviate Podcast #93! Apr 15, 2024

Hey everyone! I am SUPER excited to publish our newest Weaviate podcast with Kyle Davis, the creator of RAGKit! At a high-level, the podcast covers our understanding of RAG systems through 4 key areas: (1) Ingest / ETL, (2) Search, (3) Generate / Agents, and (4) Evaluation. Discussing these lead to all sorts of topics from Knowledge Graph RAG, to Function Calling and Tool Selection, Re-ranking, Quantization, and many more! This discussion forced me to re-think many of my previously held beliefs about the current RAG stack, particularly the definition of “Agents”. I came in believing that the best way of viewing “Agents” is an abstraction on top of multiple pipelines, such as an “Email Agent”, but Kyle presented the idea of looking at “Agents” as scoping the tools each LLM call is connected to, such as `read_email` or `calculator`. Would love to know what people think about this one, as I think getting a consensus definition of “Agents” can clarify a lot of the current confusion for people building with LLMs / Generative AI.

VetRec with David de Matheu - Weaviate Podcast #92! Mar 28, 2024

I've seen a lot of interest around RAG for X application domain, Legal, Accounting, Healthcare, .... David and Kevin are maybe the best example of this I have seen so far, pivoting from Neum AI to VetRec!

We begin the podcast by discussing the decision to switch gears, the advice given by Y Combinator, and David's experience in learning a new application domain.

We then continue to discuss technical opportunities around RAG for Veterinarians, such as SOAP notes and Differential Diagnosis!

We conclude with David's thoughts on the ETL space, companies like Unstructured and LlamaIndex's LlamaParse, advice for specific focus in ETL, and general discussions of ETL for Vector DBs / KGs / SQL.

David and Kevin have been two of my favorite entrepreneurs I've met during my time at Weaviate! They do an amazing job of writing content that helps you live vicariously through them as they take on this opportunity to apply RAG and AI technologies to help Veterinarians!

I really hope you enjoy the podcast!

Tengyu Ma on Voyage AI - Weaviate Podcast #91! Mar 20, 2024

Voyage AI is the newest giant in the embedding, reranking, and search model game!

I am SUPER excited to publish our latest Weaviate podcast with Tengyu Ma, Co-Founder of Voyage AI and Assistant Professor at Stanford University!

We began the podcast with a deep dive into everything embedding model training and contrastive learning theory. Tengyu delivered a masterclass in everything from scaling laws to multi-vector representations, neural architectures, representation collapse, data augmentation, semantic similarity, and more! I am beyond impressed with Tengyu's extensive knowledge and explanations of all these topics.

The next chapter dives into a case study Voyage AI did fine-tuning an embedding model for the LangChain documentation. This is an absolutely fascinating example of the role of continual fine-tuning with very new concepts (for example, very few people were talking about chaining together LLM calls 2 years ago), as well as the data efficiency advances in fine-tuning.

We concluded by discussing ML systems challenges in serving an embeddings API. Particularly the challenge of detecting if a request is for batch or query inference and the optimizations that go into either say ~100ms latency for a query embedding or maximizing throughput for batch embeddings.

Self-Discover DSPy with Chris Dossman - Weaviate Podcast #90! Mar 06, 2024

One of the core values of DSPy is the ability to add “reasoning modules” such as Chain-of-Thought to your LLM programs!

For example, Chain-of-Thought describes prompting the LLM with “Let’s think step by step …”. Interestingly, this meta-prompt around asking the LLM to think this way dramatically improves performance in tasks like question answering or document summarization.

Self-Discover is a meta-prompting technique that searches for the optimal thinking primitives to integrate into your program! For example, you could “Let’s think out of the box to arrive at a creative solution” or “Please explain your answer in 4 levels of abstraction: as if you are talking to a five year old, a high school student, a college student studying Computer Science, and a software engineer with years of experience in the topic”.

I am SUPER excited to be publishing our 90th Weaviate Podcast with Chris Dossman! Chris has implemented Self-Discover in DSPy, one of the most fascinating examples so far of what the DSPy framework is capable of!

Chris is also one of the most talented entrepreneurs I have met during my time at Weaviate thanks to introductions from Bob van Luijt and Byron Voorbach. Chris built one of the earliest RAG systems for government information and is now working on LLM opportunities in marketing with his new startup, Dicer.ai!

I hope you enjoy the podcast, it was such a fun one and I learned so much!

Matryoshka Embeddings with Aditya Kusupati, Zach Nussbaum, and Zain Hasan - Weaviate Podcast #89! Feb 20, 2024

Hey everyone! Thank you so much for watching the 89th Weaviate Podcast on Matryoshka Representation Learning! I am beyond grateful to be joined by the lead author of Matryoshka Representation Learning, Aditya Kusupati, Zach Nussbaum, a Machine Learning Engineer at Nomic AI bringing these embeddings to production, and my Weaviate colleague, Zain Hasan, who has done amazing research on Matryoshka Embeddings! We think this is a super powerful development for Vector Search! This podcast covers all sorts of details from generally what Matryoshka embeddings are, the challenges of training them, experiences building an embeddings API product from Nomic AI and how it ties with Nomic Atlas, Aditya's research on differentiable ANN indexes, and many more! This was such a fun one, I really hope you find it useful! Please let us know what you think!

Instructor with Jason Liu - Weaviate Podcast #88! Feb 14, 2024

Jason Liu is the creator of Instructor, one of the world's leading LLM frameworks, particularly focused on structured output parsing with LLMs, or as Jason puts it "making LLMs more backwards compatible". It is hard to understand the impact of Instructor, this is truly leading us to the next era of LLM programming. It was such an honor chatting with Jason, his experience currently as an independent consultant and previously engineering at StitchFix and Meta makes him truly one of the most unique guests we have featured on the Weaviate podcast! I hope you enjoy the podcast!

XMC.dspy with Karel D'Oosterlinck - Weaviate Podcast #87! Feb 06, 2024

Hey everyone! Thank you so much for watching the 87th episode of the Weaviate Podcast! I am SUPER excited to welcome Karel D'Oosterlinck! Karel is the creator of IReRa (Infer-Retrieve-Rank)! IReRa is one of the most impressive systems that have been built for Extreme Multi-Label Classification, leveraging the emerging paradigm of DSPy compilation! This podcast dives into all things IReRa, XMC, DSPy compilation, and applications in Biomedical NLP and Recommendation! I hope you find this useful!

Open-Source AI with Vinod Valloppillil and Bob van Luijt - Weaviate Podcast #86! Jan 23, 2024

Hey everyone! We are super excited to publish this podcast with Vinod Valloppillil and Bob van Luijt on Open-Source AI and future directions for RAG! The podcast begins by discussing Vinod's "Halloween Documents", a series of internal strategy writings at Microsoft related to the open-source software movement! The conversation continues to discuss the current state of Open-Source in AI. One of the major points Bob has been making about the business of AI models is that the models themselves are *stateless*, akin to an MP3 file. Vinod pushes back a bit on this definition and jointly it is then settled that these models neither fall into the pure stateful or stateless bucket, rather a "pre-baked" bucket -- presenting completely new opportunities to build business around software. The conversation then continues to discuss the particular details of how people are building RAG systems and many directions for how that may evolve!

DSPy and ColBERT with Omar Khattab! - Weaviate Podcast #85 Jan 15, 2024

Hey everyone! I am beyond excited to present our interview with Omar Khattab from Stanford University! Omar is one of the world's leading scientists on AI and NLP. I highly recommend you check out Omar's remarkable list of publications linked below! This interview completely transformed my understanding of building RAG and LLM applications! I believe that DSPy will be one of the most impactful software project in LLM development because of the abstractions around *program optimization*. Here is my TLDR of this concept of LLM programs and program optimization with DSPy, I of course encourage you to view the podcast and listen to Omar's explanation haha. RAG is one of the most popular LLM programs we have seen. RAG typically consists of two components of retrieve and then generate. Within the generate component we have a prompt like "please ground your answer based on the search results {search_results}". DSPy gives us a framework to optimize this prompt, bootstrap few-shot examples, or even fine-tune the model if needed. This works by compiling the program based on some evaluation criteria we give DSPy. Now let's say we add a query re-writer that takes the query and writes a new query before sending it to the retrieval system, and a reranker that takes the search results and re-orders them before handing them to the answer generator. Now we have 4 components of query writer, retrieve, rerank, answer. The 3 components of query writer, rerank, and answer all have a prompt that can be optimized with DSPy to enhance the description of the task or add examples! This optimization is done with DSPy's Teleprompters. There are a few other really interesting components to DSPy as well -- such as the formatting of prompts with the docstrings and Signature abstraction, which in my view is quite similar to instructor or LMQL. DSPy also comes with built-in prompts like Chain-of-Thought that offer a really quick way to add this reasoning step and follow a structured output format. I am having so much fun learning about DSPy and I highly recommend you join me in viewing the GitHub repository linked below (with new examples!!): Omar also discusses ColBERT and late interaction retrieval! Omar describes how this achieves the contextualized attention of cross encoders but in a much more scalable system with the maximum similarity between vectors! Stay tuned for more updates from Weaviate as we are diving into multi vector representations to hopefully support systems like this soon!

Chapters

0:00 Weaviate at NeurIPS 2023!

0:38 Omar Khattab

0:57 What is the state of AI?

2:35 DSPy

10:37 Pipelines

14:24 Prompt Tuning and Optimization

18:12 Models for Specific Tasks

21:44 LLM Compiler

23:32 Colbert or ColBERT?

24:02 ColBERT

Subjectivity in AI with Dan Shipper: AI-Native Databases #4 Dec 21, 2023

Hey everyone! Thank you so much for watching the fourth and final episode of the AI-Native Database series with Dan Shipper! This was another epic one! Dan has had an absolutely remarkable career creating and selling a company and now co-founding and working as the CEO of Every! Every is an incredibly future-looking business focused on content online, both with an amazing newsletter, community of writers and thinkers, an AI-note taking app, and more! I think Dan brings a very unique perspective to the series, as well as the Weaviate podcast broadly, because of his experience with writers and understanding how writers are going to use these new technologies! We heavily discussed the role of personality or subjectivity in AI, amongst many other topics! I really hope you enjoy the podcast, as always we are more than happy to answer any questions or discuss any ideas you have about the content in the podcast! Read writings from Dan Shipper on Every: https://every.to/@danshipper Chapters 0:00 AI-Native Databases 0:58 Welcome Dan Shipper! 1:37 GPT-4 is a Reasoning Engine 8:40 Subjectivity in LLMs 12:14 AI in Note Taking 16:38 The opinions of LLMs 25:50 Cookbooks for you 31:16 Overdrive in LLMs 34:50 Tweaking the voice of AI 40:45 Multi-Agent Personalities

Humans and AI with John Maeda: AI-Native Databases #3 Dec 20, 2023

Hey everyone! Thank you so much for watching the 3rd episode of the AI-Native Database series featuring John Maeda and Bob van Luijt! This one dives into how humans perceive AI, from Anthroaormorphization to Doomsday scenario thinking and how important understanding how AI actually work is to the engineering of these systems. Bob and John discuss the evolution of the design in tech report, 3 categories of design, and many others! I hope you enjoy the podcast! As always, we are more than happy to answer any questions or discuss any ideas you have about the content in the podcast! Links: Design in Tech Report: https://designintech.report/ 3 Kinds of Design: https://qz.com/1585165/john-maeda-on-the-importance-of-computational-design Microsoft Semantic Kernel: https://github.com/microsoft/semantic-kernel Chapters 0:00 AI-Native Databases 0:58 Welcome John Maeda! 1:35 Design in Tech Report 4:07 Anthropomorphizing AI 15:30 3 Types of Design 19:30 The ChatGPT Shift 22:58 Explaining Technology 32:54 Impact of AI on the Creative Industries 39:00 Semantic Kernel

Structure in Data with Paul Groth: AI-Native Databases #2 Dec 19, 2023

Hey everyone! Thank you so much for watching the second episode of AI-Native Databases with Paul Groth! This was another epic one, diving deep into the role of structure in our data! Beginning with Knowledge Graphs and LLMs, there are two perspectives: LLMs for Knowledge Graphs (using LLMs to extract relationships or predict missing links) and then Knowledge Graph for LLMs (to provide factual information in RAG). There is another intersection that sits in the middle of both LLMs for KGs and KGs for LLMs, which is using LLMs to query Knowledge Graphs, e.g. Text-to-Cypher/SPARQL/... From there I think the conversation evolves in a really fascinating way exploring the ability to structure data on-the-fly. Paul says "Unstructured data is now becoming a peer to structured data"! I think in addition to RAG, Generative Search is another underrated use case -- where we use LLMs to summarize search results or parse out the structure. Super interesting ideas, I hope you enjoy the podcast -- as always more than happy to answer any questions or discuss any ideas you have about the content in the podcast! Learn more about Professor Groth's research here: https://scholar.google.com/citations?... Knowledge Engineering using Large Language Models: https://arxiv.org/pdf/2310.00637.pdf How Much Knowledge Can You Pack into the Parameters of a Language Model? https://arxiv.org/abs/2002.08910 Chapters 0:00 AI-Native Databases! 0:58 Welcome Paul! 1:25 Bob’s overview of the series 2:30 How do we build great datasets? 4:28 Defining Knowledge Graphs 7:15 LLM as a Knowledge Graph 15:18 Adding CRUD Support to Models 28:10 Database of Model Weights 32:50 Structuring Data On-the-Fly

Self-Driving Databases with Andy Pavlo: AI-Native Databases #1 Dec 18, 2023

Hey everyone! Thank you so much for watching the first episode of AI-Native Databases with Andy Pavlo! This was an epic one! We began by explaining the "Self-Driving Database" and all the opportunities to optimize DBs with AI and ML at both the low-level, as well as how we query and interact with them. We also discussed new opportunities with DBs + LLMs, such as bringing the data to the model (such as ROME, MEMIT, GRACE), in addition to bringing the model to the data (such as RAG). We also discuss the subjective "opinion" of these models and many more! I hope you enjoy the podcast! As always we are more than happy to answer any questions or discuss any ideas you have about the content in the podcast! This one means a lot to me. Andy Pavlo's CMU DB course was one of the most impactful resources in my personal education, and I love the vision for the future outlined by OtterTune! It was amazing to see Etienne Dilocker featured in the ML for DBs, DBs for ML series at CMU. I am so grateful to Andy for joining the Weaviate Podcast! Links: CMU Database Group on YouTube: https://www.youtube.com/@CMUDatabaseGroup/videos Self-Driving Database Management Systems - Pavlo et al. - https://db.cs.cmu.edu/papers/2017/p42-pavlo-cidr17.pdf Database of Databases: https://dbdb.io/ Generative Feedback Loops: https://weaviate.io/blog/generative-feedback-loops-with-llms Weaviate Gorilla: https://weaviate.io/blog/weaviate-gorilla-part-1 Chapters 0:00 AI-Native Databases 0:58 Welcome Andy 1:58 Bob’s overview of the series 3:20 Self-Driving Databases 8:18 Why isn’t there just 1 Database? 12:46 Collaboration of Models and Databases 20:05 LLM Schema Tuning 23:44 The Opinion of the System 28:20 PyTorchDB - Moving the Data to the Model 33:30 Database APIs 38:15 Learning to operate Databases 42:54 Vector DBs and the DB Hype Cycle 51:38 SQL in Weaviate? 1:07:40 The Future of DBs 1:14:00 Thank you Andy!

Weaviate 1.23 Release Podcast with Etienne Dilocker! Dec 14, 2023

Hey everyone! Thank you so much for watching the Weaviate 1.23 Release Podcast with Weaviate Co-Founder and CTO Etienne Dilocker! Weaviate 1.23 is a massive step forward for managing multi-tenancy with vector databases. For most RAG and Vector DB applications, you will have an uneven distribution in the # of vectors per user. Some users have 10k docs, others 10M+! Weaviate now offers a flat index with binary quantization to efficiently balance when you need an HNSW graph for the 10M doc users and when brute force is all you need for the 10k doc users! Weaviate also comes with some other "self-driving database" features like lazy shard loading for faster startup times with multi-tenancy and automatic resource limiting with the GOMEMLIMIT and other details Etienne shares in the podcast! I am also beyond excited to present our new integration with Anyscale (@anyscalecompute)! Anyscale has amazing pricing for serving and fine-tuning popular open-source LLMs. At the time of this release we are now integrating the Llama 70B/13B/7B, Mistral 7B, and Code Llama 34B into Weaviate -- but we expect much further development with adding support for fine-tuned models, the super cool new function calling models Anyscale announced yesterday. and other model such as Diffusion and multimodal models! Chapters 0:00 Weaviate 1.23 1:08 Lazy Shard Loading 8:20 Flat Index + BQ 33:15 Default Segments for PQ 38:55 AutoPQ 42:20 Auto Resource Limiting 46:04 Node Endpoint Update 47:25 Generative Anyscale Links: Etienne Dilocker on Native Multi-Tenancy at the AI Conference in SF: https://www.youtube.com/watch?v=KT2RFMTJKGs Etienne Dilocker in the CMU DB Series: https://www.youtube.com/watch?v=4sLJapXEPd4 Self-Driving Databases by Andy Pavlo: https://www.cs.cmu.edu/~pavlo/blog/2018/04/what-is-a-self-driving-database-management-system.html

Rudy Lai on Tactic Generate - Weaviate Podcast #78! Nov 29, 2023

Hey everyone! Thank you so much for watching the 78th episode of the Weaviate podcast featuring Rudy Lai, the founder and CEO of Tactic Generate! Tactic Generate has developed a user experience around applying LLMs in parallel to multiple documents, or even folders / collections / databases. Rudy discussed the user research that lead the company to this direction and how he sees the opportunities in building AI products with new LLM and Vector Database technologies! I hope you enjoy the podcast, as always more than happy to answer any questions or discuss any ideas you have about the content in the podcast! Learn more about Tactic Generate here: https://tactic.fyi/generative-insights/ Weaviate Podcast #69 with Charles Pierse: https://www.youtube.com/watch?v=L_nyz1xs9AU Chapters 0:00 Welcome Rudy! 0:48 Story of Tactic Generate 7:45 Finding Common Workflows 19:30 Multiple Document RAG UIs 26:14 Parallel LLM Execution 32:40 Aggregating Parallel LLM Analysis 38:25 Pretty Reports 44:28 Research Agents

RAGAS with Jithin James, Shahul Es, and Erika Cardenas - Weaviate Podcast #77! Nov 20, 2023

Hey everyone, thank you so much for watching the 77th Weaviate Podcast on RAGAS, featuring Jithin James, Shahul ES, and Erika Cardenas! RAGAS is one of the hottest rising startups in Retrieval-Augmented Generation! RAGAS began it's journey with the RAGAS score, a matrix of evaluations for generation and retrieval. Generation evaluated on Faithfulness (is the response grounded in the context) as well as Relevancy (is the response useful). Retrieval is then evaluated on Precision (How many of the search results are relevant to the question?) and Recall (How many of the relevant search results are captured in the retrieved results?). Now, the super novel thing about this is that an LLM is used to determine these metrics. So we circumvent painstaking manual labeling effort with the RAGAS score! This podcast dives into the development of the RAGAS score as well as how RAG application builders should think about the knobs to tune for optimizing their RAGAS score: embedding models, chunking strategies, hybrid search tuning, rerankers, ... ?!? We also discussed tons of exciting directions for the future such as fine-tuning smaller LLMs for these metrics, agents that use tuning APIs, and long context RAG! Check out the docs here for getting started with RAGAS! https://docs.ragas.io/en/latest/getstarted/index.html#get-started Chapters 0:00 Welcome Jithin and Shahul! 0:44 Welcome Erika! 0:56 RAGAS, Founding Story 2:38 Weaviate + RAGAS integration plans 4:44 RAG Knobs to Tune 25:50 RAG Experiment Tracking 34:52 LangSmith and RAGAS 38:55 LLM Evaluation 40:25 RAGAS Agents 44:00 Long Context RAG Evaluation

Patrick Lewis on Retrieval-Augmented Generation - Weaviate Podcast #76! Nov 14, 2023

Hey everyone, I am SUPER excited to present our 76th Weaviate Podcast featuring Patrick Lewis, an NLP Research Scientist at Cohere! Patrick has had an absolutely massive impact on Natural Language Processing with AI and Deep Learning! Especially notable for the current climate in AI and Weaviate is that Patrick is the lead author of the original "Retrieval-Augmented Generation" paper!! Patrick has contributed to many other profoundly impactful papers in the space as well such as DPR, Atlas, Task-Aware Retrieval with Instruction, and many many others! This was such an illuminating conversation, here is a quick overview of the chapters in the podcast! 1. Origin of RAG - Patrick explains the build-up that lead to the RAG paper, AskJeeves, IBM Watson, conceptual shift to retrieve-read in mainstream connectionist approaches to AI. 2. Atlas - Atlas shows that a much smaller LLM when paired with Retrieval-Augmentation can still achieve competitive few-shot and zero-shot task performance. This is super impactful because this few-shot and zero-shot capability has been a massive evangelist for AI broadly, and the fact that smaller Retrieval-Augmented models can do this is massive for the economically unlocking these applications. Teasing apart some architectural details of RAG: 3. Fusion In-Decoder - Interesting encoder-decoder transformer design in which each document + the query is encoded separately, then concatenated and passed to the LM. 4. End-to-End RAG - How to think about jointly training an embedding model and an LLM augmented with retrieval? 5. Query Routers - How to route queries from say SQL or Vector DBs? (More nuance on this later with Multi-Index Retrieval) 6. ConcurrentQA - Super interesting work on the privacy of multi-index routers. For example, if you ask "Who is the father of our new CEO" - this may reveal the private information of the new CEO with the public query of their father. 7. Multi-Index Retrieval 8. New APIs for LLMs 9. Self-Instructed Gorillas 10. Task-Aware Retrieval with Instructions 11. Editing Text, EditEval and PEER 12. What future direction excites you the most? Links: Learn more about Patrick Lewis: https://www.patricklewis.io/ Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks: https://arxiv.org/abs/2005.11401 Atlas: https://arxiv.org/pdf/2208.03299.pdf Fusion In-Decoder: https://arxiv.org/pdf/2007.01282.pdf Chapters 0:00 Welcome Patrick Lewis! 0:36 Origin of RAG 5:20 Atlas 10:43 Fusion In-Decoder 17:50 End-to-End RAG 27:05 Query Routers 32:05 ConcurrentQA 37:30 Multi-Index Retrieval 40:05 New APIs for LLMs 41:50 Self-Instructed Gorillas 44:35 Task-Aware Retrieval with Instructions 52:00 Editing Text, EditEval and PEER 55:35 What future direction excites you the most?

Tanmay Chopra on Emissary - Weaviate Podcast #75! Nov 08, 2023

Hey everyone! Thank you so much for watching the 75th Weaviate Podcast featuring Tanmay Chopra! The podcast details Tanmay's incredible career in Machine Learning from Tik Tok to Neeva and now building his own startup, Emissary! Tanmay shared some amazing insights into Search AI such as how to process Temporal Queries, how to think about diversity in Retrieval, and Query Recommendation products! We then dove into the opportunity Tanmay sees in fine-tuning LLMs and knowledge distillation that motivated Tanmay to build Emissary! I thought Tanmay's analogy of GPT-4 to 3D printers was really interesting, tons of great nuggets in here! I really hope you enjoy the podcast, as always more than happy to answer any questions or discuss any ideas with you related to the content in the podcast! Chapters 0:00 Welcome Tanmay! 0:23 Early Career Story 2:02 Tik Tok 4:10 Neeva 8:45 Temporal Queries 11:40 Retrieval Diversity 17:22 Query Recommendation 23:20 Emissary, starting a company! 30:20 A Simple API for Custom Models 35:42 GPT-4 = 3D Printer?

Simba Khadder on FeatureForm - Weaviate Podcast #74! Nov 07, 2023

Hey everyone! Thank you so much for watching the 74th Weaviate Podcast feature Simba Khadder, the CEO and Co-Founder of FeatureForm! To begin, "features" broadly describe the inputs to machine learning models that they use to produce outputs, or predictions. Feature stores orchestrate the construction of features, whether that be transformations for tabular machine learning models such as XGBoost, to chunking for vector embedding inference, and now features for LLM inference in RAG. Right out of the gate, Simba really opened my eyes to the role that feature engineering plays in RAG. Further touching on this at the very end under the "Exciting future for RAG with Features" chapter, Simba further describes how we can use more advanced features to provide better context to LLMs. In addition to these insights on RAG, there are so many nuggets in the podcast, Simba is a world class professional when it comes to building distributed systems, production scale recommendation systems, and more! I learned so much from chatting with Simba, I hope you enjoy listening to the podcast! As always we are more than happy to answer any questions or discuss any ideas you have about the content in the podcast! FeatureForm: https://www.featureform.com/ Highly Recommend!! Simba Khadder at the CMU DB Seminar series: https://www.youtube.com/watch?v=ZsWa6XiBc-U FeatureForm and Weaviate demo! https://docs.featureform.com/providers/weaviate Chapters 0:00 Simba Khadder 0:35 RAG and Feature Stores 4:30 Experience building Recommendation Systems 9:47 The End-to-End Feature Lifecycle 15:08 Virtual Feature Store Orchestration 26:45 RAG Evaluation 31:27 Feature Engineering 34:15 LLM Tuning and Features 39:55 Streaming Features 51:15 Data Drift Detection 54:20 Exciting future for RAG with Features

Charles Packer on MemGPT - Weaviate Podcast #73! Nov 06, 2023

Hey everyone! I am SUPER excited to publish our 73rd Weaviate Podcast with Charles Packer, the lead author of MemGPT at UC Berkeley! MemGPT presents the "Operating System for LLMs", an incredibly exciting idea to explicitly prompt the LLM with the information that it has a limited context window and give it memory management tools to behave accordingly! This was such a fun discussion with Charles diving into all things related to the paper! I hope you enjoy the podcast!! Check out MemGPT here! https://memgpt.ai/ Chapters 0:00 Welcome Charles! 0:27 LLM Operating System 4:47 Memory Management Tools 6:50 Interrupts in LLM Applications 10:15 LLM Tools 17:45 Self-Instruct Data Creation 20:50 Cost of Experiments 24:28 Explicit Context Annotation 29:40 Recall vs. Archival Storage 33:12 Page Replacement Inspiration 38:00 Creativity in AI 43:40 Evolutionary Perspective 46:18 Inspiring Future Directions 48:45 Multi-Threaded LLM Processing

Madelon Hulsebos on Tabular Machine Learning - Weaviate Podcast #72! Nov 01, 2023

Hey everyone! Thank you so much for watching the 72nd episode of the Weaviate Podcast with Madelon Hulsebos!! Madelon is one of the world's experts on Machine Learning with Tables and Tabular-Structured Data, this was such an eye-opening conversation! We discussed all sorts of topics from the relationship of tabular data and embeddings, to searching through tables, semantic joins, more complex Text-to-SQL, using machine learning for query execution, using tabular data in search and recommendation reranking, and many more! This was easily one of the most knowledge packed episodes of the Weaviate podcast so far, please don't hesitate to leave any questions or ideas you have related to the content discussed! You can learn more about Madelon's incredible research career and publications / talks here: https://www.madelonhulsebos.com/! Papers such as GitTables are listed here! Another nice nugget form the podcast - Madelon introduced me to the BIRD-SQL benchmark which really expanded my understanding of Text-to-SQL (https://arxiv.org/pdf/2305.03111.pdf. Chapters 0:00 Welcome Madelon! 0:58 Tabular Data and Embeddings 3:10 Tabular Representation Learning 5:48 Semantic Type Detection 9:50 Pandas as an LLM Tool 11:52 Table-Based Question Answering and Text-to-SQL 19:35 Joins with Machine Learning 21:38 Query Execution with Machine Learning 22:45 Graph Neural Networks 24:07 XGBoost 28:28 Merging Tables 32:10 Fact Representation 35:50 GPT-4V and Tables 39:00 Metadata in Embeddings 42:45 Table Retrieval in Weaviate 46:25 Exciting future directions!!

Vibs Abhishek on Alltius AI - Weaviate Podcast #71! Oct 26, 2023

Hey everyone! Thank you so much for watching the 71st Weaviate Podcast with Vibs Abhishek! Vibs is the CEO and Founder of Alltius AI, as well as a professor at UC Irvine business school! In order to tame the somewhat chaotic emerging landscape of RAG and LLM applications, Alltius has settled on 3 core pillars of Knowledge, Skills, and Deployment Channels! Vibs further explained how he sees the distinction between Assistants and Agents and many more topics important to Enterprise deployment of RAG applications such as reducing hallucinations and employing classifiers to route skills and knowledge sources! I learned so much from this conversation, I hope you enjoy the podcast! Alltius KNO Plus Demo Video: https://www.loom.com/share/fcfe516b75ea4f069b1a8d6a3510fa4c?sid=5f43317f-c20b-4dd9-91d3-2cde993fd91f Chapters 0:00 Welcome Vibs 0:22 Background 2:30 Alltius’ UI for Assistants 7:15 The Knowledge Pillar 12:05 SQL Router and Intent Management 14:10 Classifying a Pipeline / Skill 17:30 Flexibility of Zero-Shot versus Fine-Tuning 21:00 The Channels Pillar 23:00 Connecting the Warehouse / Lakehouse 24:50 Assistant versus Agent 28:30 MemGPT 31:25 Offline LLM Research 35:50 Multi-Agent Role-Playing Assistants 39:25 From Clicks to Conversations 44:10 CEO / Professor and Evolution of the Field

MemGPT Explained! Oct 24, 2023

Thank you so much for watching our paper summary video on MemGPT! MemGPT is a super exciting new work bridging together concepts in how Operating Systems manage memory and LLMs! Links: Paper: https://arxiv.org/pdf/2310.08560.pdf Andrej Karpathy on Operating Systems and LLMs: https://twitter.com/karpathy/status/1707437820045062561 Run LLM Podcast with Charles Packer: https://www.youtube.com/watch?v=4aOLxPdx1Dg SciPhi: https://github.com/SciPhi-AI/sciphi/tree/main Our perspectives on Database Agents that WRITE to Vector Databases: https://weaviate.io/blog/generative-feedback-loops-with-llms Chapters 0:00 Introduction to MemGPT 2:45 MemGPT Architecture 6:15 Operating System for LLMs 11:48 Types of Context and Storage 15:42 Control Flow 18:00 Experiments 22:04 Future Work 24:46 Personal Takeaways 30:34 Thank you for watching!

Kevin Cohen on Neum AI - Weaviate Podcast #70! Oct 18, 2023

Hey everyone! Thank you so much for watching the 70th episode of the Weaviate podcast with Neum AI CTO and Co-Founder Kevin Cohen! I first met Kevin when he was debugging an issue with his distributed node utilization and have since learned so much from him about how he sees the space of Data Ingestion, also commonly referenced as ETL for LLMs! There are so many interesting parts to this from the general flow of data connectors, chunkers and metadata extractors, embedding inference, and the last leg of the mile of importing the vectors to a Vector DB such as Weaviate! I really loved how Kevin broke down the distributed messaging queue and system design for orchestrating data ingestion at massive scale such as dealing with failures and optimizing the infrastructure as code setup. We also discussed things like new use cases with quadrillion scale vector indexes and the role of knowledge graphs in all this! I really hope you enjoy the podcast, please check out this amazing article below from Neum AI! https://medium.com/@neum_ai/retrieval-augmented-generation-at-scale-building-a-distributed-system-for-synchronizing-and-eaa29162521 Chapters 0:00 Check this out! 1:18 Welcome Kevin! 1:58 Founding Neum AI 6:55 Data Ingestion, End-to-End Overview 9:10 Chunking and Metadata Extraction 14:20 Embedding Cache 16:57 Distributed Messaging Queues 22:15 Embeddings Cache ELI5 25:30 Customizing Weaviate Kubernetes 38:10 Multi-Tenancy and Resource Allocation 39:20 Billion-Scale Vector Search 45:05 Knowledge Graphs 52:10 Y Combinator Experience

Charles Pierse on Tactic Generate - Weaviate Podcast #69! Oct 04, 2023

Hey everyone! Thank you so much for watching the 69th episode of the Weaviate Podcast featuring Charles Pierse from Tactic! Tactic has recently launched their new Tactic Generate project, an incredible UI for conducting research across multiple documents. I think there is a massive opportunity to pair these prompts and LLM workflows with User Interfaces and take more of a holistic User Experience perspective. Tactic Generate has done an incredible job of that, please take a look from the link below! I had such a fun conversation catching up with Charles (Charles was our 2nd Weaviate Podcast guest!), I hope you enjoy the podcast! Tactic Generate: https://tactic.fyi/generative-insights/ Chapters 0:00 Tactic Generate 1:40 Welcome Charles! 2:38 Charles’ work at Tactic 4:40 LLMs comparing documents 9:10 LLM Chaining 17:30 Discovering LLM Chains 20:28 Moats in ML Products 28:48 Fine-Tuning vs. RAG 34:30 Fine-Tuning Search Models 39:45 Skepticism on RLHF 41:52 Gorilla, Integrations, and CRM 45:40 Query Routers 47:55 CRM and Tree-of-Thoughts 55:54 Graph Embeddings 1:02:20 Llama CPP / GGML 1:04:28 What are you looking forward to most in AI?

Weights and Biases on Fine-Tuning LLMs - Weaviate Podcast #68! Sep 20, 2023

Hey everyone! Thank you so much for watching the 68th episode of the Weaviate Podcast! We are super excited to welcome Morgan McGuire, Darek Kleczek, and Thomas Capelle! This was such a fun discussion beginning with generally how see the space of fine-tuning from why you would want to do it, to the available tooling, intersection with RAG and more! Check out W&B Prompts! https://wandb.ai/site/prompts Check out the W&B Tiny Llama Report! https://wandb.ai/capecape/llamac/reports/Training-Tiny-Llamas-for-Fun-and-Science--Vmlldzo1MDM2MDg0 Chapters 0:00 Tiny Llamas! 1:53 Welcome! 2:22 LLM Fine-Tuning 5:25 Tooling for Fine-Tuning 7:55 Why Fine-Tune? 9:55 RAG vs. Fine-Tuning 12:25 Knowledge Distillation 14:40 Gorilla LLMs 18:25 Open-Source LLMs 22:48 Jonathan Frankle on W&B 23:45 Data Quality for LLM Training 25:55 W&B for Data Versioning 27:25 Curriculum Learning 29:28 GPU Rich and Data Quality 30:30 Vector DBs and Data Quality 32:50 Tuning Training with Weights & Biases 35:47 Training Reports 42:28 HF Collections and W&B Sweeps 44:50 Exciting Directions for AI

Farshad Farahbakhshian and Etienne Dilocker on Weaviate and AWS - Weaviate Podcast #67! Sep 13, 2023

Hey everyone! Thank you so much for watching the 67th Weaviate Podcast, announcing Weaviate on the AWS Marketplace! This was one of my favorite podcasts to date with a deep dive on the details of running RAG applications in the cloud, our general understanding of LLM Fine-Tuning and RAG, as well as a really interesting discussion on VPCs and Hybrid SaaS! I hope you find the podcast useful, as always we are more than happy to answer any questions or discuss any ideas you have about the content presented in the podcast! Learn more here: https://aws.amazon.com/marketplace/seller-profile?id=seller-jxgfug62rvpxs As well as here: https://weaviate.io/developers/weaviate/installation/aws-marketplace Chapters 0:00 Welcome Farshad 0:38 Weaviate’s Journey to AWS 2:05 Retrieval-Augmented Generation and Vector DBs 3:44 Running AI in the Cloud 9:40 Fine-Tuning LLMs vs. RAG 10:30 Skill vs. Knowledge (Lawyer Example) 14:28 Continual Learning of LLMs 16:50 Searching through multiple sources 19:58 Hybrid Search controlled by LLMs 22:10 Classes versus Filters 25:00 SQL and Vector Search 25:55 Favorite RAG Use Cases 31:55 Cloud Benchmarking 37:00 Price Performance 38:20 Tuning HNSW 42:15 Horizontal Scalability on AWS Marketplace 47:00 Privacy Requirements 54:45 Weaviate Hybrid SaaS 59:00 AWS Marketplace

Hybrid SaaS in Weaviate Explained! Sep 12, 2023

Hey everyone! Here is a clip from our newest Weaviate podcast with Farshad Farahbakhshian, Gen AI specialist at AWS and Etienne Dilocker, CTO and Co-Founder of Weaviate! This podcast announces Weaviate on the AWS marketplace and is packed with info on running Weaviate in the cloud such as this clip explaining how Hybrid SaaS works! I hope you find the clip useful, we are more than happy to answer any questions you have about the content in this clip! Chapters 0:00 Quick Intro for Context 0:29 Etienne Dilocker on Hybrid SaaS

David Garnitz on VectorFlow - Weaviate Podcast #66! Sep 07, 2023

Hey everyone! Thank you so much for watching the 66th Weaviate Podcast with David Garnitz, the creator of VectorFlow! VectorFlow (open-sourced on GH and linked below) is a new tool for ingesting data into Vector Databases such as Weaviate! There is quite an interesting End-to-End stack emerging at the ingestion layer, from retrieving data from misc. sources such as Slack, Salesforce, GitHub, Google Drive, Notion, ... to then Chunking the Text (maybe with the use of Visual Document Layout parsers like what Unstructured is imagining), extracting Metadata potentially (say the "age" of an NBA player as in the Evaporate-Code+ research) -- then sending this data off to embedding model inference and unpacking that can of worms from inference acceleration to load balancing, and finally -- importing the vectors themselves to Weaviate! I learned so much from this conversation, I really hope you enjoy listening and please check out VectorFlow below! VectorFlow: https://github.com/dgarnitz/vectorflow Chapters 0:00 VectorFlow on GitHub! 0:52 Welcome David Garnitz! 1:17 Vector Flow, Founding Vision 2:00 Billions of Vectors in Weaviate! 4:20 End-to-end data importing 6:30 Metadata Extraction in Vector Database Flows 10:15 Vectorizing 100s of millions of billions of chunks 15:58 Fine-Tuning Embedding Models 23:50 Zero-Shot Models in Metadata and Chunking 36:36 Vector + SQL 42:45 Self-Driving Databases 49:23 Generative Feedback Loop REST API 51:38 GPT Cache 55:55 Building VectorFlow

Ofir Press on AliBi and Self-Ask - Weaviate Podcast #65! Aug 31, 2023

Hey everyone! Thank you so much for watching the Weaviate Podcast! I am SUPER excited to publish my conversation with Ofir Press! Ofir has done incredible work pioneering AliBi attention and Self-Ask prompting and I learned so much from speaking with him! As always we are more than happy to answer any questions or discuss any ideas you have about the content in the podcast! +Huge Congratulations on your Ph.D. Ofir! AliBi Attention: https://arxiv.org/abs/2108.12409 Self-Ask Prompting: https://arxiv.org/abs/2210.03350 Ofir Pres on YouTube: https://www.youtube.com/@ofirpress Chapters 0:00 Welcome Ofir Press 0:41 Large Context LLMs 12:38 Quadratic Complexity of Attention 19:12 AliBi Attention, Visual Demo! 24:53 Recency Bias in LLMs 28:57 RAG in Long Context LLM Training 36:27 Self-Ask Prompting 46:07 Chain-of-Thought and Self-Ask 50:47 Gorilla LLMs 58:42 New Directions for New Training Data

Shishir Patil and Tianjun Zhang on Gorilla - Weaviate Podcast #64! Aug 30, 2023

Hey everyone! Thank you so much for watching the 64th Weaviate Podcast with Shishir Patil and Tianjun Zhang, co-authors of Gorilla: Large Language Models Connected with Massive APIs! I learned so much about Gorilla from Shishir and Tianjun, from the APIBench dataset to the continually evolving APIZoo, how the models are trained with Retrieval-Aware Training, Self-Instruct Training data and how the authors think of fine-tuning LLaMA-7B models for tasks such as this, and many more! I hope you enjoy the podcast! As always I am more than happy to answer any questions or discuss any ideas you have about the content in the podcast! Please check out the paper here! https://arxiv.org/abs/2305.15334 Chapters 0:00 Welcome Shishir and Tianjun 0:25 Gorilla LLM Story 1:50 API Examples 7:40 The APIZoo 10:55 Gorilla vs. OpenAI Funcs 12:50 Retrieval-Aware Training 19:55 Mixing APIs, Gorilla for Integration 25:12 LlaMA-7B Fine-Tuning vs. GPT-4 29:08 Weaviate Gorilla 33:52 Gorilla and Baby Gorillas 35:40 Gorilla vs. HuggingFace 38:32 Structured Output Parsing 41:14 Reflexion Prompting for Debugging 44:00 Directions for the Future

Nils Reimers on Cohere Search AI - Weaviate Podcast #63! Aug 17, 2023

Hey everyone! Thank you so much for watching the 63rd Weaviate Podcast, I couldn't be more excited to welcome Nils Reimers back to the podcast!! Similar to our debut episode together, we began by describing the latest collaboration of Weaviate and Cohere (episode 1, new multilingual embedding models; episode 2, rerankers!), and then continued into some of the key questions around search technology. In this one, we discussed the importance of temporal queries and metadata extraction, long document representation, and future directions for Retrieval-Augmented Generation! I hope you enjoy the podcast, as always I am more than happy to answer any questions or discuss any ideas you have about the content in the podcast! Thank you so much for watching! Learn more about Cohere Rerankers and how to use it in Weaviate here: https://weaviate.io/developers/weaviate/modules/retriever-vectorizer-modules/reranker-cohere Chapters 0:00 Introduction 1:30 Cohere Rerankers 7:02 Dataset Curation at Cohere 10:30 New Rerankers and XGBoost 14:35 Temporal Queries 17:55 Metadata Extraction from Unstructured Text Chunks 21:52 Soft Filters 24:58 Chunking and Long Document Representation 38:00 Retrieval-Augmented Generation 45:40 Retrieval-Aware Training to solve Hallucinations 49:50 Learning to Search and End-to-End RAG 54:35 RETRO 59:25 Foundation Model for Search

Atai Barkai on PodcastGPT - Weaviate Podcast #62! Aug 09, 2023

Hey everyone! Thank you so much for watching the 62nd Weaviate Podcast with Atai Barkai! We are stepping into the meta with this one for a podcast about podcasts! Podcasts are one of the biggest opportunities of new technologies, starting with Whisper's ability to transcribe audio to text and advances with speaker diarization, .. the question to be explored is, What Vector Database and LLM applications can we build with this data?! What is the future of podcasting with these new technologies?! I had so much fun discussing all these ideas with Atai! As always we are more than happy to answer any questions or discuss any ideas you have about content discussed in the podcast! Thank you so much for watching! Chapters 0:00 Welcome Atai! 1:04 TawkitAI and PodcastGPT! 2:20 Chat with Podcast PodcastGPT - https://www.podcastgpt.ai/ Tawkit AI - https://twitter.com/tawkitapp Weaviate Podcast Search Demo! https://github.com/weaviate/weaviate-podcast-search

Rohit Agarwal on Portkey - Weaviate Podcast #61! Aug 03, 2023

Hey everyone! Thank you so much for watching the 61st episode of the Weaviate Podcast! I am beyond excited to publish this one! I first met Rohit at the Cal Hacks event hosted by UC Berkeley where we had a debate about the impact of Semantic Caching! Rohit taught me a ton about the topic and I think it's going to be one of the most impactful early applications of Generative Feedback Loops! Rohit is building Portkey, a SUPER interesting LLM middleware that does things like load balancing between LLM APIs, and as discussed in the podcast there are all sorts of opportunities for this kind of space whether it be routing to tool-specific LLMs, different cost / accuracy requirements, or multiple models in the HuggingGPT sense. It was amazing chatting with Rohit, this was the best dive into LLMOps I have personally been apart of! As always we are more than happy to answer any questions or discuss any ideas you have about the content in the podcast! Check out portkey here! https://portkey.ai/blog Chapters 0:00 Introduction 0:24 Portkey, Founding Vision 2:20 LLMOps vs. MLOps 4:00 Inference Hosting Options 7:05 3 Layers of LLM Use 8:35 LLM Load Balancers 12:45 Fine-Tuning LLMs 17:08 Retrieval-Aware Tuning 21:16 Portkey Cost Savings 23:08 HuggingGPT 26:28 Semantic Caching 32:40 Frequently Asked Questions 34:00 Embeddings vs. Generative Tasks 35:30 AI Moats, GPT Wrappers 39:56 Unlocks from Cheaper LLM Inference

Patrice Bourgougnon on WPSolr - Weaviate Podcast #60 Aug 02, 2023

Hey everyone! Thank you so much for watching the 60th Weaviate podcast with Patrice Bourgougnon! Patrice is the creator of WPSolr, integrating AI search capabilities with Wordpress and Woocommerce. Patrice is one of the most active contributors to Weaviate, filing issues and poking holes in new releases! Patrice shared incredible feedback on Weaviate and how he sees the state of Vector Databases and Search! As always, we are more than happy to answer any questions or ideas you have about the content discussed in the podcast! Thanks for watching! Chapters 0:00 Introduction 0:45 Vector Databases and Wordpress 4:50 Weaviate Client Languages 10:00 Inference and Database Container Management 21:30 Business Opportunities for Search in Production 26:40 Testing Search Performance, “Something to sleep on” 30:50 Zero-Shot Model Ability 36:05 Make LLMs Stateful 43:46 Chatbots and Search Boxes 44:55 Mixing Models in Applications 47:00 BM25 vs. Vector Search in RETRO RAG

Andriy Mulyar on Nomic AI, Atlas, and GPT4All - Weaviate Podcast #58 Jul 18, 2023

Hey everyone! Thank you so much for watching the 58th episode of the Weaviate Podcast! I am SUPER excited to welcome Andriy Muylar! Andriy is the Co-Founder of Nomic AI, a company fresh off a $17M series A raise! Nomic has created some incredible products such as Atlas and GPT4All! I was really impressed by Andriy's vision of the state and forecasted evolution of these topics! I hope you enjoy the podcast! As always, we are more than happy to answer any questions or discuss any ideas you have about the content discussed in the podcast! Integration Tutorial for Weaviate and Nomic AI Atlas! https://docs.nomic.ai/vector_database.html This example worked for me if you want to clone it with the podcast transcription dataset: https://github.com/weaviate/weaviate-podcast-search/blob/main/atlas-visualizer.py Check out Nomic AI here! https://home.nomic.ai/blog Chapters 0:00 Congrats Nomic and Weaviate Integration! 2:35 Welcome Andriy Mulyar! 3:05 Founding Story of Nomic AI 6:55 Understanding Massive Scale Text Data 10:14 Topic Modeling 16:30 Monitoring Model Training

Charles Frye on Full Stack Deep Learning - Weaviate Podcast #57! Jul 13, 2023

Hey everyone! Thank you so much for watching the 57th Weaviate podcast with Charles Frye! Charles is an educator at Full Stack Deep Learning, one of the world's top courses on Deep Learning with lectures available on YouTube (link below)! This was one of the most thorough Weaviate podcasts published so far, covering all sorts of topics around the evolution of Deep Learning! Particularly we discussed the Retrieval-Augmented Generation stack with Vector Databases and Zero-Shot Large Language Models and how that compares to more conventional machine learning workflows and the MLOPs stack! I really enjoyed chatting with Charles and am more than happy to answer any questions or discuss any ideas you have about the content in the podcast! Thank you so much for listening! Check out Full Stack Deep Learning! https://fullstackdeeplearning.com/ Full Stack Deep Learning on YouTube! https://www.youtube.com/@The_Full_Stack Chapters 0:00 Welcome Charles Frye! 0:52 Charles’ journey into Deep Learning 3:00 Weights & Biases and MLOps 5:30 Retrieval-Augmented Generation Stack 8:58 Data Engines and AI Products 13:50 Fine-Tuning 16:35 Information Retrieval Techniques 20:10 RAG as Tool Use and RETRO 23:33 Gorilla and Fine-Tuned Tool Use 27:36 Text-to-SQL Tool Use 30:46 Generative Data Augmentation 33:05 LLM generated queries for embeddings 38:04 Long-Tail and Data Imbalance 41:45 LoRA LLM Fine-Tuning 44:50 Eigenvectors and Disentaglement 50:00 LLM for Each User 55:00 Embedding Visualization and ML Observability 58:40 GPU Utilization 1:05:05 Discord Q&A Bot App 1:16:10 Data Schema Design 1:21:25 Graph and Vector Databases 1:28:35 Future Directions in AI

Etienne Dilocker on Weaviate 1.20 - Weaviate Podcast #56! Jul 12, 2023

Chapters 0:00 Weaviate 1.20!!! 0:40 Multi-Tenancy 35:36 PQ Rescoring 47:20 Re-Ranking, AutoCut, Rank Fusion 58:58 Cloud Monitoring Metrics

Aleksa Gordcic - Weaviate Podcast #55! Jul 05, 2023

Hey everyone! Thank you so much for watching the 55th episode of the Weaviate Podcast with Aleksa Gordcic! This episodes dives into Aleksa's incredible story from Deep Learning YouTube to DeepMind and now creating Ortus! We dived into all sorts of topics, I loved hearing about the latest updates on Ortus and how Aleksa is sees the current state of AI development! We are more than happy to answer any questions or discuss any ideas you might have about the content in the podcast! Thanks so much for watching! Check out Ortus here! - https://www.ortusbuddy.ai/welcome Chapters 0:00 Introduction 1:08 Deep Learning YouTube 5:40 DeepMind 9:40 Ortus 19:50 LangChain and LlamaIndex 23:10 Software 2.0 and Full Stack DL 29:20 Training Embedding Models 32:23 Text Chunking for Vector DBs 34:35 Visual Information in YouTube 38:15 Simulating Conversations 42:46 Aidan Gomez Quote on Synthetic Data 44:40 Tree of Thoughts 47:40 New Ortus Features 49:00 Embedding Marketplace 54:00 Personal Organization

Stephanie Horbaczewski and Gunjan Bhattarai on Vody - Weaviate Podcast #53! Jun 22, 2023

Chapters 0:00 Introduction 0:38 Founding Story of Vody 8:15 Custom Embedding Models 12:42 Movie Genre Vectors 13:42 Classification and Contrastive Learning 15:45 Foundation Model Tuning 21:13 Multimodal Generative Models 25:08 Training Embedding Models 33:20 Tabular Data Ranking Models 36:00 RoomGPT 41:36 Diversity in Recommendations 48:25 Future Directions in Multimodal AI 51:15 Open-Source 55:45 Keeping up with Vody!

Yana Welinder on Kraftful - Weaviate Podcast #52! Jun 14, 2023

Hey everyone, thank you so much for watching the 52nd episode of the Weaviate Podcast with Yana Welinder! Yana is the Founder and CEO of Kratful (https://www.kraftful.com/). Kratful is an incredibly interesting "ChatGPT but for Product Research" -- curating specific skills for Product Managers into a collection of prompts. We discussed all sorts of things from the latest innovations in LLMs to the ChatGPT marketplace and product management, I really hope you enjoy the podcast!

Greg Kamradt and Colin Harmon on LLM Agents - Weaviate Podcast #51 Jun 07, 2023

Hey everyone, thank you so much for watching the 51st episode of the Weaviate Podcast with Greg Kamradt and Colin Harmon! Greg and Colin are both entrepreneurs in the space of new AI tools powered by LLMs! This podcast is about keeping up with the evolution of LLM Agents from AutoGPT to connecting LLMs with Vector Databases or Wolfram Alpha, as well as the ChatGPT Marketplace, Personalized LLMs, Private LLMs, and many more! I think there are so many interesting nuggets from this podcast, thank you so much to Greg and Colin for joining, really enjoyed this one! Data Independent: https://www.youtube.com/@DataIndependent Greg Kamradt on Twitter: https://twitter.com/GregKamradt Nesh: https://hellonesh.io/ Colin Harmon on LinkedIn: https://www.linkedin.com/in/coluha/ Colin Harmon Blog: https://colinharman.substack.com/ Colin Harmon at Haystack US 2023: https://www.youtube.com/watch?v=LO3U5iqnTpk Chapters 0:00 Introduction 0:42 Backgrounds 2:43 Defining “LLM Agents” 6:12 Data-Aware LLMs 13:04 Tool Use 13:38 ChatGPT API vs. Marketplace 17:40 Personalized LLMs, LLM for Greg 19:20 PrivateGPT 25:14 AutoGPT and Chain-of-Thought Prompting 32:30 Few-Shot Examples 35:30 Early AI Signals and Open-Source 43:10 Multi-Agent LLMs 47:14 Fine-Tuning and Long Input Lengths 52:20 Greg’s LLM Wishlist Hierarchy 53:15 Keeping up with Greg and Colin!

Retrieving Texts based on Abstract Descriptions Explained! Jun 02, 2023

This video explores a new paper exploring the use of summarization chains to represent long texts and use (original text, summary) pairs for optimizing text embeddings models! Here are 3 main takeaways I think everyone working with Weaviate may get value from: 1. Understanding of Summary Indexing and the Prompts (as well as Prompt Chains) used to build them. 2. Continued development of LLM-generated data for search -- creating (full text, summary) pairs gives you (1) data to build a summary index with as mentioned, (2) data to compare different embedding models with, and (3) data to train your own embedding model. 3. Tournament style evaluation with human annotators -- the top 5 retrieved texts from one model are concatenated with the top 5 from another model, these 10 are given to human annotators to pick 5 and this is how the authors are reporting the performance of their models rather than traditional benchmarks. This m ay be a more productive evaluation technique for most real world search applications. Thank you so much for watching, here are some links mentioned in the video! Retrieving Texts based on Abstract Descriptions: https://arxiv.org/abs/2305.12517 Weaviate Blog - Combining LangChain and Weaviate: https://weaviate.io/blog/combining-langchain-and-weaviate Weaviate Blog - Generative Feedback Loops: https://weaviate.io/blog/generative-feedback-loops-with-llms Jerry Liu in Llama Index Blog - A New Document Summary Index for LLM-powered QA Systems: https://medium.com/llamaindex-blog/a-new-document-summary-index-for-llm-powered-qa-systems-9a32ece2f9ec Learning to Retrieve Passages without Supervision (Spider): https://arxiv.org/pdf/2112.07708.pdf Weaviate Blog - Analysis of Spider - https://weaviate.io/blog/research-insights-spider Chapters 0:00 Introduction 0:13 Quick Overview 7:30 How to use in Weaviate! 7:50 Background 12:08 Motivation 14:20 Prompts Used 18:14 More Details of training 21:12 Human Evaluation Study 22:40 My Takeaways from the Paper

Kapa AI with Emil Sorensen and Finn Bauer - Weaviate Podcast #50! May 31, 2023

Hey everyone, thank you so much for watching the 50th (!!!) Weaviate Podcast with Emil Sorensen and Finn Bauer from Kapa AI! Are you curious about taking either your, or your company's, specific information and putting into a Vector DB + LLM system? Emil and Finn are doing this at the highest level, taking the documentation of software companies like Weaviate and building these LLM-augmetnted assistant systems for them. This podcast takes a complete tour from Data Ingestion to Cleaning, Chunking, LLM latency, and emerging trends in LLMs such as cheap fine-tuning with LoRA or Long Context Windows such as GPT-4 32K, MPT-7B 65K, or Anthropic Claude's 100k. I learned so much from speaking with Emil and Finn! Please let us know any questions you have or ideas you would like to discuss! Check out Kapa here! https://www.kapa.ai/ Chapters 0:00 Welcome Emil and Finn! 0:42 Origin Story of Kapa 2:08 Data Ingestion 5:10 Data Cleaning 6:20 Slack / Discord / Forum Ingestion 9:05 Testing Models on Support QA 11:14 Selling Kapa to Weaviate and friends 12:37 Hallucinations in LLMs 14:06 Trends in Open-Source LLMs 15:20 Long Input LLMs (32K, 65K, 100K, …) 16:54 Retrieval-Augmentation for Long Input LLMs 18:08 Fine-Tuning LLMs 23:00 As much or as refined content as possible? 24:40 Adding Docs from Integrations 26:15 Generative Feedback Loops 29:00 What in AI excites you the most?

Neurosymbolic AI in Search with Professor Laura Dietz - Weaviate Podcast #49! May 25, 2023

Hey everyone, thank you so much for watching the 49th episode of the Weaviate Podcast!! This podcast features Professor Laura Dietz from the University of New Hampshire! I came across Dr. Dietz's tutorial at ECIR on Neuro-Symbolic Approaches for Information Retrieval and am so grateful that she was interested in joining the Weaviate Podcast! I learned so much about Neurosymbolic Search, especially around the role of Entity Linking and Entity Re-Ranking -- as well as the topic of Knowledge Graphs and Vector Search. We also discussed Prof. Dietz and collaborators latest perspectives paper on Large Language Models for Relevance Judgment. TLDR this describes the idea of using LLMs to either generate synthetic queries for documents or say annotate the relevance for query, document pairs. We discussed this kind of idea with Leo Boytsov and his work on InPars, and have presented Promptagator on past episodes of the Weaviate Air show. Although this idea comes with a lot of potential, Dr. Dietz explains the potentials for bias and poor judgements, as well as generally diving more into the details of this kind of idea! I really hope you enjoy the podcast, we are more than happy to answer any questions you might have about these ideas, or discuss any of your ideas! Thanks so much for watching! Check out Laura Dietz's Publications here: https://scholar.google.com/citations?user=IIXpJ8oAAAAJ&hl=en&oi=ao ECIR 23 Tutorial: Neuro-Symbolic Approaches for Information Retrieval: https://www.cs.unh.edu/~dietz/appendix/dietz2023neurosymbolic.pdf Please check this paper out below, I think this is a severely underrated work in the Search / Information Retrieval community: Perspectives on Large Language Models for Relevance Judgment: https://arxiv.org/pdf/2304.09161.pdf Chapters 0:00 Introduction 0:15 Neurosymbolic Search 4:50 Entity Parsing and Vector Semantics 10:56 Query Intent Understanding 15:35 Knowledge Graphs and Vector Search 17:37 Symbolic Re-Ranking 22:10 ColBERT and Entity Ranking 26:25 Example - South America and Zika Virus IR 29:15 Knowledge Graph Query Languages with LLMs 35:10 We need more Knowledge Graphs!! 37:30 PrimeKG from Harvard BMI 39:40 Filtered Vector Search 42:20 LLM Entity Linking - “The” example 47:30 Cross Encoder Entity Focus? 48:25 Perspectives on LLMs for Relevance Judgments 55:28 Spectrum of Human-Machine Collaboration for Labeling 57:30 Use LLM to Create Relevance Labeling Interfaces 1:02:30 Importance for Weaviate 1:03:45 12 Authors’ 3 Conclusions 1:04:40 IR Research Community Challenge 1:06:55 Query Generation for Weaviate Users 1:13:05 Clustering Queries 1:17:30 Final Thoughts

Unstructured with Brian Raymond - Weaviate Podcast #48! May 23, 2023

Hey everyone, thank you so much for watching the 48th episode of the Weaviate Podcast!! This is a SUPER exciting one, welcoming Brian Raymond the CEO / Founder of Unstructured! Unstructured is a perfect complimenting technology for Weaviate, helping people get their Unstructured data into Weaviate! The podcast dives into the nuances of this task, but it generally revolves around Unstructured's abstraction of Partitioning, Cleaning, and Staging! Unstructured is making groundbreaking innovations on using Visual Document Layout models for Partitioning, for example saying that this part of the PDF is the header, body, image caption, and so on. Cleaning then describes removing pesky details like whitespaces or odd characters. Staging then describes the transformations of say formatting a text chunk with it's metadata into the JSON for a Weaviate object upload! I really hope you find this podcast interesting! We are publishing a blog post as well showing an example of how to use Unstructured to get PDF data into Weaviate, please please check that out and let us know if it works for your data and how we can improve it! This blog post can be found on weaviate.io and we will be managing discussions around it both in the Weaviate slack, as well as Unstructured! Thank you so much for listening! Check out Unstructured here! https://www.unstructured.io/ Chapters 0:00 Welcome Brian!! 0:27 What is Unstructured? 5:42 Why now? New Advancements in Unstructured 8:02 Thoughts on Data Connectors Hub 10:55 PDFs to Weaviate with Unstructured 13:53 State-of-the-Art in OCR and Document Parsing 16:10 How to get the data from Weaviate.io? 18:06 Foundation Models from Unstructured 20:45 Evaporate-Code+ 23:15 CSV, Parquet, JSON transformations in Staging 25:08 Cleaning Bricks 28:02 Visual Document Examples 30:45 Text Chunking with Metadata 33:25 Knowledge Graphs with Goldman Sachs example 39:10 LLM Hallucinations 42:10 Announcements from Brian!

ChatArena with Yuxiang Wu - Weaviate Podcast #47! May 17, 2023

Hey everyone, thank you so much for watching the Weaviate podcast! I am so excited about this episode! ChatArena is a software framework for multi-agent chat games. There are quite a few interesting applications of this, firstly we can use this kind of system to evaluate the intelligence of an LLM based on how intelligent it sounds in conversation with another LLM! Another interesting idea is to have the LLM impersonate people such as Lex Fridman or Sam Altman and simulate conversations between these people -- retrieving from their digital content to facilitate the impersonation. I thought there was so many interesting ideas in this podcast, please let us know what you think! Links: ChatArena on GitHub (please give it a star!) - https://github.com/chatarena/chatarena Twitter thread from Yuxiang describing the launch of ChatArena - https://twitter.com/YuxiangJWu/status/1643633046208249856 Chapters 0:00 Welcome Yuxiang! 0:38 What is ChatArena? 2:38 Impersonating People with LLMs 4:58 Weaviate and ChatArena 8:14 Generative Feedback Loops 11:10 Chat Games 16:30 Scientific Peer Review Discussions 20:05 Code Repos and Multi-Agent LLMs 23:05 Scaling Multi-Agent LLMs 25:16 Role Evolution in Startups 26:00 Evolution of Multi-Agent RL Research 29:22 AlphaGo and MCTS Text Generation 36:55 Hallucination in Role Maintenance 41:15 Evaluating LLMs with ChatArena 45:40 ChatGPT Marketplace and Tool Use 50:30 Upcoming work from Yuxiang and ChatArena!

HyperDB with John Dagdelen, Bob van Luijt, and Etienne Dilocker - Weaviate Podcast #46! May 10, 2023

Hey everyone! Thank you so much for watching the Weaviate Podcast! This is pretty novel episode featuring both Weaviate Co-Founders Bob van Luijt and Etienne Dilocker! This is also extremely novel because we are featuring a competitor vector database, HyperDB! John Dagdelen is the founder of HyperDB which is a hyper-fast local vector database for use with LLM Agents. Now accepting SAFEs at $135M cap. HyperDB: https://github.com/jdagdelen/hyperDB More seriously, John has produced an incredible body of research - https://scholar.google.com/citations?user=TiCS5FEAAAAJ&hl=en&oi=ao. John's work on Scientific Literature Mining for Materials Science literature has played an enormous role in my personal education of this technology and what it is capable. Please also follow John on twitter @jmdagdelen. Chapters 0:00 Introduction 0:26 HyperDB! 3:58 Initial Discovery of Vector Dos 15:00 Search Engine versus Databases 18:40 Scientific Literature Mining 21:42 Structured Information Extraction 27:47 Generative Feedback Loops

Generative Feedback Loops with Bob van Luijt - Weaviate Podcast #45! May 05, 2023

Hey everyone! Thank you so much for watching the Generative Feedback Loops Podcast! We have also created a blog post and GitHub repository for more information! Chapters 0:00 Bob the Podcast Host 1:20 Retrieval-Augmented Generation 4:10 Hallucination in LLMs 6:15 Solving Hallucination with RLHF 7:44 LLM Monster - Reasoning and Knowledge 10:12 Feedback Loops 11:00 Hands-on Code Demo 26:00 Demo Analysis from Bob and Connor 30:35 Star Wars Wes Anderson Generated Video 34:12 Multimodal Vector Databases 36:00 Speculative Design Theory Links: John Schulman - Reinforcement Learning from Human Feedback: Progress and Challenges: https://www.youtube.com/watch?v=hhiLw5Q_UFg Colin Nesh (HaystackUS 2023 slide deck) - Ground is NOT all you need, Stop hallucinations & defects in generative search: https://docs.google.com/presentation/d/1uycLEUeRuF8A85Uso_A3OU6EF-qq4aYswWVxkPWHBKI/edit#slide=id.p Generative Starwars video source - https://twitter.com/CuriousRefuge/status/1652412004626497536 Speculative Design Theory - https://readings.design/PDF/speculative-everything.pdf Aggregation Theory - https://stratechery.com/aggregation-theory/

Weaviate 1.19 Release with Etienne Dilocker - Weaviate Podcast #44! May 04, 2023

Hey everyone! Thank you so much for watching the Weaviate 1.19 release podcast! We have all sorts of cool new features, in addition to the database and module features, I really want to encourage readers to see the `groupBy` search discussed at 14:32, quite an interesting idea for improving search performance! Chapters 0:00 Welcome Etienne! 0:38 gRPC API 9:50 Generative Cohere 14:32 groupBy search 19:33 Bitmap or BM25 index tuning 22:20 Additional Tokenization Options 24:05 Tunable Consistency

Erika Cardenas, Roman Grebennikov, and Vsevolod Goloviznin on Recommendation and Metarank - Pod #43! Apr 12, 2023

Thank you so much for watching the 43rd episode of the Weaviate Podcast with Roman Grebennikov and Vesvolod Goloviznin from Metarank, as well as Erika Cardenas from Weaviate! This podcast is a masterclass on Ranking models, additionally touching on the connection between Search and Recommendation. Learning-to-rank is an exciting idea where we use models that produce more fine-grained relevance scores than the offline indexing techniques of vector search and bm25, however with the tradeoff of the speed of these inferences. Romand and Vsevolod touched on another extremely interesting part of these ranking models which is the estimation of features such as Click-through-Rates and how they use streaming technology to do this. I learned so much from this podcast about the directions in ranking, I hope you enjoy it as well! As always, we are more than happy to answer any questions or discuss any ideas with you! In reflecting on this podcast, Erika and I wrote up our latest thoughts on Ranking Models in a Weaviate blogpost, check it out here if interested: https://weaviate.io/blog/ranking-models-for-better-search. Chapters 0:00 Welcome Everyone! 0:40 Recommendation with Weaviate 4:20 Metarank - Founding Story 8:20 Ranking MLOps 9:52 User Friendliness Perspective 15:10 Retrieval vs. Ranking 17:45 Ranking Optimization 25:20 Multi-Vector Object Representations 27:55 Click-through-Rate Feature Streaming 33:06 Weaviate Properties vs. Feature Stores 40:06 Cold-Start Recommendation Problem 46:04 Ranklens Demo - RecSys Datasets 52:02 Cross Encoders

Ethan Steininger on Mixpeek and the AI Landscape - Weaviate Podcast #42! Apr 05, 2023

Thank you so much for watching the 42nd episode of the Weaviate Podcast! Ethan Steininger is the founder of Mixpeek, an intelligence layer that sits on top of your S3 bucket, so you can search and analyze unstructured data at scale. Ethan has also created Collie with the headline of "Enter your website and Collie will fetch every asset, then give you an embedded search bar that wows your users". Ethan began the podcast by describing his background at MongoDB and integrating the database with full text search functionality. Ethan then presented the founding vision of Mixpeek and some of the most outstanding problems with adapting the latest AI technologies to solve business problems. This lead us to discuss a massive range of topics around the AI landscape from the Llama / Alpaca models to ChatGPT Plugins, the paradigm shift in coding and serverless GPUs. I really enjoyed speaking with Ethan about all these things, I hope you enjoy listening! We would more than happy to discuss any ideas you have with you or answer any questions, thanks again for watching! Chapters 0:00 Welcome Ethan Steininger! 0:50 Entry into Search from MongoDB 6:45 Founding Vision of Mixpeek 10:15 Data Ingestion 13:45 ChatGPT Plugins 16:25 Paradigm shift in Coding with GPT-4 18:54 Alpaca Models 22:42 Tuning LLMs with Retrieval 31:45 Adding Structure to Code Repo Search 35:06 Re-Ranking / Learning-to-Rank 43:30 AGI Monopoly 49:10 Hybrid Search! Zero-Shot + BM25 54:20 Open-Source Business 59:35 Serverless GPUs 1:11:18 Ethan’s Advice for Stress Management 1:13:00 Existential AI Fear Links: Mixpeek - https://mixpeek.com/ Collie - https://collie.ai/ An Open-Source, Personalized Generative Model Framework - https://esteininger.medium.com/an-open-source-personalized-generative-model-framework-6df865de51bf Teaching GPT-4 to write code from research papers - https://esteininger.medium.com/teaching-gpt-4-to-write-code-from-research-papers-889a880fb4f0 The Need for an AI Content Verification Layer - https://esteininger.medium.com/the-need-for-an-ai-content-verification-layer-10be9379b354 Building the ML Stack of the Future - https://esteininger.medium.com/building-the-ml-stack-of-the-future-d66c8a8b566a Vertical Integration is Key to Winning the AI Race - https://esteininger.medium.com/vertical-integration-is-key-to-winning-the-ai-race-44c8e4bd3b30

Dennis Xu on Mem and LLMs! - Weaviate Podcast #41 Mar 29, 2023

Chapters 0:00 Welcome Dennis Xu! 0:30 Founding Vision of Mem 4:18 Personalized Embeddings 6:02 GPT-4, How will this change everything? 11:00 Writing code with LLMs 13:18 Embeddings at Mem 17:10 Structure in Vector Search 19:10 Zero-Shot vs. Fine-Tuned Models 25:05 Ranking Models and LLM Distillation

Weaviate 1.18 Release Podcast - Weaviate Podcast #40! Mar 07, 2023

Chapters
0:00 Weaviate 1.18!!!
0:32 Bitmap Indexing!
11:40 HNSW PQ
25:33 Cursor API
30:03 Filters in Hybrid Search
32:55 WAND Scoring
40:35 Replication
49:10 Building a Database in Golang
1:00:55 Thank you!

Floris Hoogenboom on OpenVerkiezingenNL - Weaviate Podcast #39 Mar 06, 2023

Check out the website here! https://openverkiezingen.nl/

Leo Boystov on Information Retrieval Science - Weaviate Podcast #38 Mar 01, 2023

Hey everyone! Thank you so much for watching the 38th episode of the Weaviate podcast! This episode features Leo Boystov, an expert in Information Retrieval technology! We discussed a very wide range of topics from an overview of IR methods such as BM25, Neural Bi-Encoder and Cross-Encoder rankers, and a super exciting new work Leo has co-authored on using Large Language Models to generate training data for Neural Ranking models titled "InPars-Light: Cost-Effective Unsupervised Training of Efficient Rankers." We also discussed Leo's work on Non-Metric Space Search, the challenge of long document ranking, Robustness in Generalization Testing, and ended with some thoughts on Hybrid Rank Fusion. I really hope you enjoy the podcast, more than happy to answer any questions you have or clarify anything!
In-Pars Light: Cost-Effective Unsupervised Training of Efficient Rankers - https://arxiv.org/abs/2301.02998
Google Scholar Leo Boystov - https://scholar.google.com/citations?...
Chapters
0:00 Introduction
1:08 Information Retrieval Research
25:20 Ranker Inference Requirements
40:40 Non Metric Space Search
52:38 Code Libraries for IR Research
59:40 Long Document Ranking
1:07:00 Robustness Generalization
1:15:40 Hybrid Rank Fusion

GPT Index and Weaviate with Jerry Liu and Bob van Luijt - Weaviate Podcast #37 Feb 22, 2023

Hey everyone! Thank you so much for watching the 37th episode of the Weaviate podcast! This episode discusses some of the ideas behind GPT Index. GPT Index presents really exciting ideas about how we use LLMs to index our data and then traverse these data structures. We began the podcast by discussing the origins of the tool and the ideas behind the Tree Index. We then discussed generalizing these trees to graphs and whether we are headed to the Knowledge Graph 2.0. Another really interesting topic we covered is the inference cost of building and traversing LLM indices like this! I really hope you enjoy this podcast I think these are some of the most cutting edge ideas in AI and Search!

Check out GPT Index (now LlamaIndex here - https://gpt-index.readthedocs.io/en/l...)

Chapters

0:00 Introduction

0:18 Origin Story of GPT Index

2:22 GPT Tree Index

5:53 Search Examples - Podcast Clips

11:22 Knowledge Graph 2.0?

16:05 LLM Writing Data to DB

20:18 Weaviate Classes and Index Hierarchy

23:53 Subindices vs. Tool Use

28:50 Inference Requirements for GPT Index

35:53 Design of GPT Index

37:40 Impact of Cheaper LLMs for this

40:02 Name Change for GPT Index?

42:04 Llama Hub

45:07 Relationship in Software Stack

48:15 Extension to Multimodal, e.g. Vision-Language

LangChain and Weaviate with Harrison Chase and Bob van Luijt - Weaviate Podcast #36 Feb 15, 2023

Hey everyone! Thank you so much for watching the 36th episode of the Weaviate podcast! This episode continues on the marriage between LLMs and Semantic Search, welcoming back Weaviate CEO and Co-Founder Bob van Luijt! Enter LangChain and its creator, Harrison Chase, providing the glue between LLMs and tools, such as semantic search. LangChain provides a set of abstractions around chaining multiple language model calls with different prompts, strategies for overcoming the 4096 token limit, and connecting LLMs with their tools. LangChain Hub is a collection of these chains if you want to check it out yourself! Huge thank you to Harrison and Bob for joining the podcast, this was such an information packed podcast with some great predictions for the future of LLMs + Vector Databases!

Check out LangChain here! https://langchain.readthedocs.io/en/latest/

Chapters

0:00 Welcome
0:14 Origin Story of LangChain
1:27 What are LLM Chains?
4:00 Adding Weaviate Search
7:30 LLM Orchestration and Tool Use
11:24 Extension to Multi-Modal
14:00 Natural Language Interaction with Software
20:36 Will Prompt Engineering Last?
21:00 More on Tool Use
25:47 Favorite Prompts
29:54 Temperature in LLMs
31:00 Reasoning and Knowledge
32:50 LLM as Router
35:50 Model Diversity
39:45 No GPUs before PMF
41:35 Virality of LangChain
43:40 Future of LangChain

Bob van Luijt on Generative Search with Weaviate - Weaviate Podcast #35 Feb 07, 2023

This podcast debuts a huge new release from Weaviate... the generate module! The generate module is a new API in Weaviate that facilitates passing YOUR data from the Weaviate database to ChatGPT. This enables ChatGPT to become knowledgeable about your particular business or interests! Here is a great snippet from Bob around the 43 minute mark that describes how this kind of LLM technology is changing the world of database technology, "Yeah so, what I’m really excited about and this is something that it’s just so funny right because if you see it, you have this huge epiphany. I’ve always been thinking of working with these models on input. Right so that they we can solve the problem of not having 100% keyword based search, so that we can have semantic search, image search, and those kind of things. I saw that as this beautiful uniqueness coming from a vector search engine or vector search database. So now what we’re adding is not only the input in the database but the output. So we’re basically saying we’re going to give you relevant information coming from the database, but that’s not per se stored inside the database. That’s new! I mean, just think about the most used databases in the world, Postgres, or MySQL, those kind of databases. It only outputs what’s in there. It makes sense. Because that’s how you use it. But now what we’re saying, is that’s fine you can do that, but also it can give you information, give you data that’s generated based on a task or prompt that you’re giving it. Having databases that make sense of it at input and generate new relevant content if that’s something you want as a user is amazing, and it’s just getting started. We should do this podcast like a half a year from now again and see how it's evolved because this is just too exciting man.". I really hope you enjoy the podcast, we are more than happy to answer any questions or help you get started with Weaviate!

Our Mad Journey of Building a Vector Database in Go - Weaviate at FOSDEM 2023 Feb 06, 2023

Chapters
1:00 Introduction
1:26 Why does the world need yet another database?
3:57 Memory Allocations
9:40 Delayed Decoding
16:05 SIMD
22:04 Demo Time!
24:38 Mad at Go?
26:00 Audience Questions

Dmitry Kan on Neural Search Frameworks - Weaviate Podcast #34 Jan 25, 2023

I am so excited to host Dmitry Kan on the Weaviate Podcast!! Dmitry is a world class expert on emerging trends in search technology! This podcast reflects on Dmitry's latest characterization of the field, the Neural Search Pyramid. This describes the different components involved with building a Deep Learning-powered Search experience from the Approximate Nearest Neighbor index algorithms, to Database functionality, LLM orchestration, Vectorization optimization, Data preprocessing, User Interface, and many more! We also concluded the podcast with an interesting debate around renaming "Vector Search" to something else that reaches a broader audience. I really hope you enjoy the podcast, thank you so much for listening! Please see the links below to Dmitry's recent content and the Weaviate Podcast Search App!

Links:

Dmitry's Keynote at Haystack Europe 2022, Where Vector Search is Taking Us - https://www.youtube.com/watch?v=2o8-dX__EgU

Dmitry's latest blog post on Neural Search Frameworks: A Head-to-Head Comparison - https://dmitry-kan.medium.com/neural-search-frameworks-a-head-to-head-comparison-976aa6662d20.

Search through this episode of the Weaviate Podcast! - https://github.com/weaviate/weaviate-podcast-search

Chapters

0:00 Neural Search Pyramid Visual

0:40 Weaviate Podcast Search!

1:35 Welcome Dmitry!!

2:02 Where is Vector Search taking us?

5:40 Retail and Search

11:02 Neural Search Frameworks

17:10 Data Preprocessing, e.g. PDF to Text / OCR

24:15 Vectorizing Data

31:18 ANN Index and Database Entanglement

37:25 Hardware Accelerators for Vector Search

46:02 Reader Layers, Q&A, Ranking, …

51:20 ChatGPT in Neural Search Frameworks

1:03:40 Search Result Summarization with ChatGPT

1:12:55 User Interfaces for Neural Search

1:26:30 Renaming “Vector Search”

1:46:10 Thank you Dmitry!!

Nils Reimers on Cohere Embedding Models Jan 11, 2023

Weaviate podcast #33.

Thank you so much for watching the 33rd Weaviate Podcast! This episode features one of the heroes of Deep Learning for Search, Nils Reimers! Nils' work on SentenceBERT is one of the foundational works for applying Deep Representation Learning to text search. This is the idea that personally inspired me to work in this field. Having seen the successes of Contrastive Representation Learning for Computer Vision, I was mind-blown by the possibility of this for NLP and text search. In addition to the scientific foundation, the software development of the Sentence Transformers library and BEIR benchmarks has been enormously impactful! It was an honor getting to ask Nils the questions I have about these things, from the role of Data Quality to Intent, Sparse Vectors, Long Document Encoding, Distribution Shift, and many more. I really hope you enjoy the podcast! We are so excited about the Cohere Multilingual embedding model and can't wait to see what else comes out of Cohere and their amazing team!
Cohere Multilingual ML Models with Weaviate: https://weaviate.io/blog/2022/12/Cohe...
Nils Reimers: https://scholar.google.com/citations?...
Mentioned in the podcast,
Cross-Encoders: https://weaviate.io/blog/2022/08/Usin...
How to choose a Sentence Transformer from HuggingFace: https://weaviate.io/blog/2022/10/How-...
Chapters
0:00 Cohere X Weaviate
0:22 Welcome Nils Reimers!
1:18 Origin Story
3:15 Learning Text Embeddings
6:54 Positive and Negative Sampling in Contrastive Learning
13:32 1 Billion Pairs for Text Embedding Optimization
15:44 Impact of Data Quality
18:40 New Cohere Multilingual Model!
24:50 Challenge of Debugging Multilingual Models
28:30 Intent in Search
30:40 Thoughts on ColBERT
33:50 Sparse Vectors in Search
36:17 Long Documents and Multi-Discourse
43:40 Entity Parsing in Query Understanding
46:08 Unknown Words and Distribution Shift
50:07 Re-Vectorizing with Fine-Tuning
53:07 More on Search Interfaces and Intent in Search
55:15 Thank you Nils!

Sam Bean, Zain Hasan, and John Trengrove on You.com and Spark Jan 09, 2023

Weaviate Podcast #32.

Thank you so much for watching the Weaviate podcast! We are super excited to host Sam Bean from You.com! As well as welcome Zain Hasan and John Trengrove to the Weaviate podcast for the first time! Sam begins by describing You.com, and then we dive into the Weaviate Spark Connector that Sam played a massive role in creating. I thought this was such a masterclass in the Spark big data technology; John, Sam, and Zain are all data engineering pros and I've never learned more about a new technology from a podcast than this one. I really hope you enjoy listening to it, please let us know any questions or ideas you have. Also, please see Zain's blog post on "The Details Behind the Sphere Dataset in Weaviate" - https://weaviate.io/blog/2022/12/deta.... This provides great detail on exactly how to use the Spark connector in Weaviate! In this case for a billion-scale dataset upload!!!
Chapters
0:00 Thanks for Watching!
0:18 Welcome Zain and John
0:28 Welcome Sam Bean, You.com!
1:48 Search Interface and Search Apps / Widgets
3:40 Searching through Specific Websites
4:00 Origin Story of You.com
6:53 How did you come across Weaviate?
8:33 Text, Image, Audio Search
10:28 What do you use Spark for?
14:20 Datasets used with Weaviate
16:14 Creating a Spark Connector to Weaviate
21:05 Adding Streaming support
22:50 Vectorizing Data at You.com
27:15 More on ONNX + Spark
29:52 Performance Questions, Spark + Weaviate
34:35 Parquet for HuggingFace Dataset Files
34:54 What is Parquet? Spark Pushdown Filters Explained
39:04 Similar to HDF5?
39:45 Spark for Extracting Ranking Features
43:25 Hybrid Search
46:48 Collecting Search Relevance Data
51:07 Thank you for watching! Thanks Sam, Zain, and John!

Weaviate 1.17 Release with Etienne Dilocker and Parker Duckworth Dec 21, 2022

Weaviate Podcast #31.

Weaviate 1.17!! This is a massive release for Weaviate, debuting Replication, Hybrid Search, BM25, Faster Startup and Import Times, as well as other fixes! Replication and Hybrid Search are two massive features for Weaviate, and we really hope you enjoy the description of them from the podcast. Please also check out the Weaviate 1.17 release blog post for more information as well - https://weaviate.io/blog/2022/12/Weaviate-release-1-17.html!

This is also a very special podcast as we welcome Parker Duckworth for the first time to the podcast! Parker gave an excellent explanation of Replication and unpacked some of the questions we are seeing around Ref2Vec! Thank you so much for listening to the podcast! Please check out the newest version of Weaviate!

Chapters

0:00 Weaviate 1.17! Welcome Parker!

0:28 From Italy to 1.17

2:04 Replication work in Italy

3:58 Replication Details

6:28 Use Cases of Replication

13:12 Product Engineering

16:24 Hybrid Search

21:30 Open Question around Hybrid Search

23:15 Rank Fusion 24:00 BEIR Benchmarks

27:28 What is Ref2Vec?

29:08 Bipartite Graph Ref2Vec Example

29:30 Graphs in Weaviate

34:25 Ref2Vec Cascading Updates

37:45 Custom Aggregation Functions in Ref2Vec

39:08 Adding Recency Bias in Ref2Vec

41:18 Startup Time Improvements

41:50 Batch Latency Improvement

Bob van Luijt, Chris Dossman and Marco Bianco on the future of search Dec 14, 2022

Weaviate Podcast #30.

Chapters

0:00 The future of search!
0:42 Welcome Marco and Chris!
4:28 Solving Hallucination with External Memory LLMs
8:16 Bob van Luijt on Weaviate and LLMs, Collaborations
14:48 What we have is not yet what the technology is capable of
16:45 Everything is Search!
18:55 The Magic of Machine Learning
20:30 Asking follow up questions
22:28 Meaning in LLMs and RLHF
27:10 How ChatGPT is Evangelizing the Technology
29:45 What is the future of search from a user perspective?
34:38 Integration with Existing Businesses
35:20 Impact on Creativity
37:37 Data Visualization from Natural Language Questions
39:00 Thought experiment - is this notebook an extension of the brain?
42:20 More on “always on” interface
43:42 General vs. Specific Intelligence
45:25 Software Business Impact
48:12 Open-Source Models
49:25 Finding Niches
56:30 Pride in Humans + AI
57:20 Exploring more Prompts
59:20 Personalized Embedding Key
1:02:40 Concluding Thoughts

Matthijs Douze on Quantization and FAISS Nov 30, 2022

Weaviate Podcast #29. Hey everyone, thank you so much for watching another episode of the Weaviate podcast! This episode features Matthijs Douze, one of the most talented and accomplished scientists we've hosted on the Weaviate podcast! Matthijs has pioneered the use of Product Quantization to compress vector representations and enable even faster and more efficient approximate nearest neighbor vector search. Matthijs told an incredible story about the history of this research, from searching from SIFT vectors for Computer Vision Search applications like real-time CD Cover album search to the problems facing modern IVF-PQ systems and the use of PQ in graph-based HNSW search. This is also a very special episode as Abdel Rodriguez makes his debut on the Weaviate podcast to discuss Weaviate's efforts in integrating PQ support and the unique challenges with this algorithm and the incremental updates required for a Vector Database. On this topic, Etienne Dilocker also returned to discuss the topic of Vector Database vs. Library with Matthijs, who is one of the lead developers of the Faiss library. This was a really information-heavy podcast, please don't hesitate to ask us any questions or present any of your ideas! Thanks again for listening!

Maarten Grootendorst on BERTopic Nov 17, 2022

Weaviate Podcast #28. Thank you so much for watching the 28th Weaviate Podcast! This episode features Maarten Grootendorst, developer of the BERTopic python library and an active evangelist of this exciting cluster analysis technology, (Maarten has written some incredible articles here - https://medium.com/@maartengrootendorst)! In this podcast, Maarten did an incredible job explaining how BERTopic works, with particular details such as k-Means clustering vs. HDBSCAN, Semi-Supervised topic modeling, Dynamic topic modeling, and many more! I was amazed at Maarten's expertise in the miscellaneous details of these algorithms! We are extremely excited about adding BERTopic to Weaviate, please see this proposal if interested in contributing to the discussion: https://github.com/semi-technologies/...!

Michael Goin on Neural Magic Oct 26, 2022

Weaviate Podcast #27. Thank you so much for watching the 27th episode of the Weaviate Podcast! This is truly one of my favorite podcasts we have published so far, I think the way Weaviate and Neural Magic fit together is really exciting! Michael did an amazing job explaining the concepts behind how Neural Magic delivers and tests inference acceleration, as well as the vision for the future of Deep Learning with Sparsity and CPU inference. I really hope you enjoy the podcast, more than happy to answer any questions or entertain any ideas/discussion! Thanks again for watching! Weaviate users can begin using Neural Magic's text vectorization pipeline as a custom text2vec-transformers docker image here - cshorten/experimental-text2vec-neuralmagic. Please note this is an experimental build and we will be releasing the full integration with thorough testing very soon!

Jonathan Frankle on MosaicML Cloud Oct 19, 2022

Weaviate Podcast #26. Thank you so much for watching the 26th episode of the Weaviate Podcast! This is another really special episode! Jonathan Frankle is one of the world's experts in Deep Learning and is making incredible advances at MosaicML in efficient Deep Learning training. The headline event is the release of MosaicML Cloud and a set of new cost estimates for GPT language models at different scales (linked below). Jonathan explains that these numbers are a baseline and he predicts they could get to as low as $100K as they seek opportunities for efficiency optimizations. This story has already played out in the realm of ResNet ImageNet training as MosaicML has demolished expectations of how fast we can train these models and it seems highly likely they will do the same for large language model costs. Jonathan and I also discussed the general space of Language Models and their applications, especially discussing their role as Databases in things like the Weaviate Vector Search Engine. We also discussed Self-Ask, Chain-of-thought Prompting, and tool use in Language Models. I had an awesome time picking Jonathan's brain about these topics and I hope you all enjoy the podcast, more than happy to answer any questions or entertain any ideas / discussion! Thanks again for watching! Blog post: GPT-3 Quality for less than $500K - https://www.mosaicml.com/blog/gpt-3-q...

Erik Bernhardsson and Etienne Dilocker on Vector Search in Production. Oct 06, 2022

Weaviate Podcast #25. Thank you so much for watching the 25th episode of the Weaviate Podcast! This is a really special episode with Erik Bernhardsson! Erik is one of the early thought leaders on Approximate Nearest Neighbor (ANN) Search, creating the ANNOY library at Spotify. Erik shared incredible insights about vector search at Spotify such as the role of Offline and Online Machine Learning inference and the role of multi-stage re-ranking pipelines. Erik has also done massively impactful work on benchmarking ANN algorithms! We really hope you enjoy the podcast and would be thrilled to answer any questions you have about the conversation topics! Thanks again for watching!

Weaviate v1.15 Release with Etienne Dilocker and Dirk Kulawiak Sep 08, 2022

Weaviate Podcast #24. Weaviate v1.15 Release! Thank you so much for checking out the Weaviate podcast -- here is a summary of what is new in Weaviate 1.15: 1. Cloud-native backups – allows you to configure your environment to create backups – of selected classes or the whole database – straight into AWS S3, GCS or local filesystem 2. Reduced memory usage - we found new ways to optimize memory usage, reducing RAM usage by 10-30%. 3. Better control over Garbage Collector – with the introduction of GOMEMLIMIT we gained more control over the garbage collector, which significantly reduced the chances of OOM kills for your Weaviate setups. 4. Faster imports for ordered data – by extending the Binary Search Tree structure with a self-balancing Red-black tree, we were able to speed up imports from O(n) to O(log n) 5. More efficient filtered aggregations – thanks to optimization to a library reading binary data, filtered aggregations are now 10-20 faster and require a lot less memory. 6. Two new distance metrics – with the addition of Hamming and Manhattan distance metrics, you can choose the metric (or a combination of) to best suit your data and use case. 7. Two new Weaviate modules – with the Summarization module, you can summarize any text on the fly, while with the HuggingFace module, you can use compatible transformers from the HuggingFace 8. Other improvements and bug fixes – it goes without saying that with every Weaviate release, we strive to make Weaviate more stable – through bug fixes – and more efficient – through many optimizations. Please check out this awesome blog post from Sebastian Witalec and the team describing these further - https://weaviate.io/blog/2022/09/Weav....

Ori Ram on Learning to Retrieve Passages without Supervision Aug 31, 2022

Weaviate Podcast #23. Thank you so much for watching the 23rd episode of the Weaviate Podcast! This episode dives into a new technique for Self-Supervised retrieval in NLP with some incredible results!

Yaoshiang Ho on Masterful AI Aug 10, 2022

Weaviate Podcast #22. Thank you so much for watching the 22nd Weaviate Podcast with Yaoshiang Ho! Yaoshiang is a Co-Founder of Masterful AI, a company doing incredible work in the Computer Vision model training and deployment space (https://www.masterfulai.com/). I really hope you enjoy this podcast, Yaoshiang and I went deep into some of the cutting edge Computer Vision algorithms such as Noisy Student, SimCLR, and Barlow Twins -- as well as the broader topic of Semi-Supervised Learning in which we have a small labeled dataset and a large unlabelled dataset. I am so excited about model training tools like Masterful and the integration with the Weaviate Vector Search Indexes and other Database features! More than happy to answer any questions/host any discussion on topics mentioned in the podcast! Please hit the like and subscribe to support more content like this!

Laura Ham on Weaviate User Experience Jul 27, 2022

Weaviate Podcast #21. Thank you for watching the 21st Weaviate Podcast with Laura Ham! Laura Ham has worked on Weaviate at SeMI Technologies for a little over 5 years. She has had a heavy influence on all things from the GraphQL User Experience design to the Graph data model, and the creation of educational content! I really enjoyed this podcast, please see the list of topics under “chapters”! Here are some examples of recent coding tutorial videos Laura has made on “How to add custom modules to Weaviate” and integrations of Weaviate with Jina AI and Haystack.

Tuana Celik on Question Answering with Haystack Jul 13, 2022

Weaviate Podcast #20. Tuana Celik, a Developer Advocate at Deepset, presented many exciting ideas around Question Answering! We began with her Game of Thrones Question Answering Demo on HuggingFace Spaces and continued to discuss all topics QA from Extractive to Abstractive, benefits of Retrieve-then-Read, and Zero-Shot Generalization, to give a quick preview. For our Weaviate users, please check out this demo from Laura Ham on how to use Haystack QA in tandem with the Weaviate Vector Search Database: https://www.youtube.com/watch?v=Bkoza.... I really hope you enjoy this podcast, please don't forget to check the Chapters to see if any topics appeal particularly to what you are working on! Please also see the links below with referenced content in the podcast!

Etienne Dilocker on Weaviate v1.14 Release! Jul 08, 2022

Weaviate Podcast #19. SeMI Technologies Co-Founder and CTO Etienne Dilocker returns to the Weaviate podcast to describe what's new with Weaviate v1.14! Please see the chapter outlines if you would like to skip ahead to the update most relevant to you! Please also see this blog post lead by Sebastian Witalec describing the new changes to Weaviate! Weaviate v1.14 Blog Post: https://weaviate.io/blog/2022/07/Weav...

Vincent D. Warmerdam on Applications of Nearest Neighbor Search Jun 28, 2022

Weaviate Podcast #18. Thank you for watching the 18th Weaviate Podcast with Vincent D. Warmerdam! Vincent is an engineer at Spacy working on exciting tools such as Prodigy! Vincent describes how nearest neighbor search can aid in tasks such as Data De-Duplication and Data Labeling! Vincent shared many interesting ideas from representations of text, challenges with annotator disagreement, lessons from hosting data labeling workshops to train data scientists, and many more!

Kyle Lo on Scientific Literature Mining May 31, 2022

Weaviate Podcast #17. Thank you for watching the 17th Weaviate Podcast with Kyle Lo! Vector Search enables us to find semantically similar items in large collections. Scientific Literature Mining is an extremely interesting case of this where we search through enormous collections of scientific papers to find evidence and ideas. Kyle has an extremely impressive resume in this application domain, tackling tasks such as Question Answering, Summarization, Fact Verification, and more! We really hope you enjoy the lessons Kyle describes from building these systems. Further, we hope that this inspires excitement in your own Vector Search applications!

Etienne Dilocker on ANN Benchmarks May 24, 2022

Weaviate Podcast #16. ANN Benchmarks are a tool for evaluating the performance of in-memory approximate nearest neighbor algorithms. Etienne Dilocker, the CTO of SeMI Technologies, the company behind Weaviate shares some insight knowledge about this interesting topic.

Maximilian Werk on Jina AI's Neural Search Framework May 03, 2022

Weaviate Podcast #15. Weaviate is used as a database for Jina AI's Neural Search Framework. In this podcast, Maximilian Werk, Engineering Director at Jina AI, will talk about all things related to this neural search framework together with Connor Shorten. Also, Maximilian will give a Jina Example Walkthrough... Enjoy!!

UNC research team on VL Adapter for Efficient CLIP Transfer Apr 26, 2022

Weaviate Podcast #14. Thanks for watching the Weaviate podcast! Our 14th episode welcomes Yi-Lin Sung, Jaemin Cho, and Professor Mohit Bansal, a research team from UNC! Our guests present their work on VL Adapter, a technique to achieve full fine-tuning performance while only updating 4% of original parameters!! This is an incredibly interesting finding for the sake of cost-effective tuning of Vision and Language models based on CLIP. We additionally discussed topics around compression bottlenecks in neural architectures, V&L datasets, and the tricky question of compositional generalization. If you are curious about using CLIP in Weaviate, please check out this text-to-image search example with Unsplash images and a React frontend!

Data Science with Rick Lamers from Orchest Apr 05, 2022

Weaviate Podcast #13. Rick Lamers, CEO, and Founder of Orchest.io. Orchest is a tool targeted at data scientists and this software simplifies building data pipelines.

Jonathan Frankle, Research Scientist in Deep Learning Mar 29, 2022

Weaviate Podcast #12. Please check out Composer from MosaicML! https://github.com/mosaicml/composer Jonathan Frankle is the Chief Scientist at MosaicML and a PhD student in Machine Learning at MIT. Jonathan is the first author of “The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks” which won an ICLR best paper award. You can learn more about Jonathan Frankle here: http://www.jfrankle.com/. Here is an explanation of how to use the Python library of Composer: https://www.youtube.com/watch?v=Xi_5w...

CEO Han Xiao From Jina AI Mar 15, 2022

Weaviate Podcast #11. You can now use Weaviate as the document store for DocumentArray in Jina AI. We had the pleasure to talk with their CEO Han Xiao. See the timestamps below what it is all about, or check out the recap from Henry AI!

Yury Malkov and Etienne Dilocker about HNSW in Vector Search and Weaviate Mar 10, 2022

Weaviate Podcast #10. A guided conversation about HNSW by Connor Shorten between Yury Malkov, Staff ML Engineer at Twitter and the co-inventor of HNSW, and Etienne Dilocker, the co-founder of Weaviate. Check the timestamps below!

Karen Beckers about The Role of Vector Search in eCommerce Mar 03, 2022

Weaviate Podcast #9. Karen Beckers, Data Scientist from Squadra Machine Learning Company, gives insightful information about how to use vector search in eCommerce in this podcast with Connor Shorten. Some topics are image-based datasets, vector search for data scientists, the future of eCommerce, and many more! See the timestamps below for more information.

Brady Neal about Causal Inference in Vector Search Feb 28, 2022

Weaviate Podcast #8. Brady Neal from Oogway talks with Connor Shorten from Henry AI Labs about causal inference and many more. See the timestamps below to check out what this podcast is all about.

Arvind Neelakantan of OpenAI • Embeddings API in Weaviate Feb 11, 2022

Weaviate Podcast #7. Arvind Neelakantan, Research Lead at Open AI, talks with Connor Shorten about their newly released embeddings API, his work at Open AI, the integration into Weaviate, and more.

How Zencastr Searches through their Podcast Transcriptions with Weaviate Feb 02, 2022

Weaviate Podcast #6. Alex Cannan, a Machine Learning engineer at Zencastr, talks with Connor Shorten about a really exciting use case of applying search to look through podcast transcription. Topics discussed are the need for fine-tuning, building your own vector database versus Weaviate, data privacy for Deep Learning applications, and many more!

How The Knowledge Management Bot Katie leverages Weaviate Jan 21, 2022

Weaviate Podcast #5. Katie is a knowledge management bot, continuously improving, self-learning, and trained by humans. Under the hood, Katie is powered by the Weaviate vector search engine, during this podcast, Katie's Michael Wechner will talk about all things vector search and more!

On Deepset's Haystack and how they leverage The Weaviate Vector Search Engine Jan 11, 2022

Weaviate Podcast #4. NLP frameworks like Deepset's Haystack are powerful tools to help data scientists and software engineers work with the latest and greatest in natural language processing. In this interview, Malte Pietsch will be talking about Haystack and how they leverage the Weaviate vector search engine as a persistent storage engine for their data and vector representations.

A Vision for The Future of Vector Search Dec 20, 2021

Weaviate Podcast #3. Join ‪Connor Shorten and Bob van Luijt (SeMI Technologies) for the third Weaviate vector search engine Podcast. During the show, they will be discussing use cases, the GraphQL API, knowledge graphs, Weaviate as a product, vector search engine use cases, and a vision for the future of vector search.

How Keenious uses Weaviate to Enable Semantic Search through 60M+ Academic PUBs Dec 13, 2021

Weaviate Podcast #2. Join ‪Connor Shorten (Henry AI Labs) and Charles Pierse (Keenious) for the second Weaviate vector search engine Podcast. During the show, they will be discussing how Keenious uses Weaviate and broader, all things NLP!

Community and Weaviate Core Update Dec 05, 2021

Weaviate Podcast #1. Join ‪Connor Shorten and Etienne Dilocker (SeMI Technologies) for the first Weaviate Podcast. During the show, they will be discussing Weaviate's horizontal scalability features in the v1.8.0 release and a wide variety of topics surrounding the Weaviate Slack channel.

Our TOPPODCAST Picks

Follow Us

Stay Connected

Related Podcasts

Links

Stay Connected