Logseq Jupyter mirror: best of both worlds

Customization Look what I built
1
  • Logseq is great for organizing thoughts
    • I discovered Logseq in mid-2022 and explored in some depth Notion, Capacities, Anytype, Affine, Gitbook, and a few others, and I’m back to Logseq. For a long time, I think.
    • Previously, Logseq users have built in JS and python execution (via pyodide)
    • Logseq is useful for writing code, but execution options are limited (it’s not and shouldn’t be a Swiss army knife)
  • Jupyter is great for running code (primarily python)
    • Searching for information inside Jupyter notebooks is awkward, and retrieving text or code is worse, due to JSON structure and having to open the notebook in Jupyter
    • Various issues with source control are partially addressed by converting to plain text documents, esp. markdown
    • Authoring UI and UX is not as rich as in dedicated markdown editors
  • Logseq combined with Jupyter is going to be great for organizing thoughts and code

I’ve been tinkering with the idea of a two-way mirror that would cast Jupyter .ipynb into Logseq .md, and vice versa. Any other markdown flavor can be supported as well. Some features I’m targeting:

  • Batch conversions
    • :white_check_mark: IPYNB → MD
    • MD → IPYNB
  • Real-time, birectional updates, should be easy enough with python’s watchdog. I’m not well-versed in JS but probably there are options.
  • :white_check_mark: Support for Logseq properties block
  • In addition to markdown and code inputs, it would be useful to capture certain outputs
    • :white_check_mark: plain text and numbers
    • :white_check_mark: HTML tables (from SQL queries via Pandas/Polars)
    • images (e.g. from matplotlib/seaborn) - would be nice but not a priority for me
  • Support for non-local (cloud) files and Jupyter instances (long-term vision)
  • Early (static) prototype is shown here
  • Development hurdles: how can one enable auto-refresh on the Logseq side? Currently, external edits to Logseq files require manual refresh, and active page is reset to today’s journal.
  • How would you use it?
  • If you want it, would you like to collaborate?

previous threads on the subject of jupyter+logseq: Search results for 'jupyter' - Logseq

ljmirror1
ljmirror12550×1693 237 KB

2

Welcome.

What execution options do you miss in particular?

3

Hi - thanks for all your work on this subject.

I need to cover several main use cases for myself:

  1. work development (with credentials in dotenv), most often I’m circling between writing modules in VSCode and testing them in Jupyter with autoreload; Jupyter servers are often remote
  2. trying out new stuff and contributing to OS packages (mainly local development)
  3. writing code-related content

Logseq’s main role would be to search across private and published text and code.

I’ve looked into existing options and concluded that they don’t serve my uses cases well enough:

  1. pyodide python (@mentaloid)
    pyodide library support so far is limited to some of the most popular libraries; I work in life sciences and need various less-known packages of varying quality.
  2. linking to a running jupyter kernel (@aarimond)
    here there are a lot more options for environment management and the full power of jupyter; still, giving up Jupyter UI entirely seems too radical
  3. did I miss anything?
4
  • Searching is a read-only process, not needing bidirectional updates.
    • Likewise for querying and reporting.
  • Searching inside Jupyter notebooks may be awkward, but searching inside Logseq has not been a nice experience either.
    • This may change in the coming database version.
  • I’m so far missing the intended type of marriage.
    • Sounds like this is mostly Jupyter (both execution and UI), with just a little of Logseq.