Deep and Native Integration between Logseq and LLMs (such as ChatGPT)

I probably don’t need to stress how powerful LLM can be in boosting productivity with knowledge work. I am aware there is a plugin that allows you to summarize content with LLM (ChatGPT) already, but I think there should be more to that. Here are two features that I imagine would make Logseq a killer app (if not already) for knowledge management:

  1. Block level embedding. Each block can be converted into a text embedding – be it OpenAI embedding or other open source embedding model – and be stored along with the block itself (probably as a metadata field or in a local vector database). This would allow much more powerful “Unlinked Reference” on a block level, as it connects related concepts based on meaning rather than textual similarity.

Text embedding also enables you to chat with your entire knowledge base (just like ChatPDF.com, but with your entire knowledge base)

  1. Speaking of ChatPDF, I think there should be a chat interface built natively into Logseq (e.g., as the sidebar). This should not only allow you to chat with the focal page, but also allows you to chat with attached document (say, attached PDF files). Since PDF files are immutable, embeddings can be easily cached in the edn file or as a separate file.

These are just some initial thoughts. I am not sure if this is on the roadmap already, but I totally see how this would elevate my productive to another level!

Given the incredible pace of developpement of AI frameworks, I think it would be better to integrate with langchain. Langchain would be the one having to deal with the new models, new ideas etc. Basically just integrate with the langchain ecosystem instead of doing anything more complicated.

1 Like

I believe that the integration of LLM will be independent of the AI framework. For instance, my proposal does not depend on any particular AI vendor or framework, whether it be OpenAI, LLaMa, or Langchain. What is crucial is to have the infrastructure in place to facilitate the integration of AI.

1 Like

Something similar to this plugin in Obsidian:

4 Likes