Command Line to Export Pages and Data

I’d love to be able to access Export public pages, Export as JSON, etc via a command line interface.

The ability to automate exporting would pair really nicely with the logseq/publish project. I’d use this to create a custom autopublishing blog sourced from my own knowledge base.

This would also pair nicely with GitHub Actions, so we could generate public pages every time we commit.

3 Likes

I would like to second this … something like

logseq --graph=/path/to/graph --export-graph --as=public --destination=/path/to/html/dir

would be very nice. Note that I need --graph=/path/to/graph as I might be working on several graphs.

I would use this from cron so that I can update a local HTML repository … I have other users that need to see a read-only view of my graph.

2 Likes

--graph=/path/to/graph and --destination=/path/to/html/dir are self explanatory. But what would the --export-graph and --as=public options do?

@emu Hi. GitHub - pengx17/logseq-publish: Logseq Publish Action provides the ability to autopublish blogs via a github action. It’s used by over 200 blogs. Does this solve your use case?

@cldwalker the repo from pengx17 uses a full docker container and a screen reader automation tool to click publish in the Logseq desktop app… basically using a sledgehammer to push in a thumbtack… this would be handled so much more properly with a simple CLI command.

I see you have a depreciated repo that uses JS: GitHub - logseq/publish

1 Like

It looks like there is a page for the Logseq API: https://plugins-doc.logseq.com/

I didn’t see one for publish, but looking into the code base I found 103 (js/window.apis.exportPublishAssets.. in src/main/frontend/handler/export.cljs … and the function defn- handle-export-publish-assets in [src/electron/electron/core.cljs]

So it looks like there may already be a publish window API in use. How can this be made available to the main API?

Then, what is the best way to call it? I see there are slash commands made for lines, but how about a CLI command to run the plugin APIs? Or if this is a cross-platform problem, maybe a plugin that accepts API commands to the server with curl (an external API).

For example, the plugin API for Rename Page is here: https://github com/logseq/logseq/blob/7aa40af68/libs/src/LSPlugin.ts#L619

Or deletePage directly above it deletePage: (pageName: BlockPageName) => Promise<void> that takes a page name. And the logseq.Git.execCommand. I dont see these functions in the rest of the Typescript. Where does it go to execute them? Are they buried in the libraries? Thanks

EDIT:
I started a new post thread to explore how to make changes to Logseq: How do the plugin APIs call functions in the app?

I also found the codebase overview which describes the design, along with the refactoring post at the bottom: logseq/CODEBASE_OVERVIEW.md at master · logseq/logseq · GitHub

I was able to build the electron app from source (logseq/develop-logseq.md at master · logseq/logseq · GitHub), but digging into the clojure its a bit hard to follow where the functions are located. I started looking at src/main/frontend/publishing.cljs, and it looks like this is where the plugin API would tie into.

@xaq_asdqwe What specific publish functionality do you want to use a CLI for that GitHub - pengx17/logseq-publish: Logseq Publish Action is unable to do? At some point we could provide a CLI that publishes the current export if there are enough compelling use cases

Unfortunately the development of logseq/publish stopped over a year ago. It is a good amount of work
to convert and support all the logseq concepts to a static version that is separate from the current SPA (web version). If you or someone else is interested in working on this, please let us know.

@cldwalker Not OP but for me what Logseq provides now is enough but you are forced to use the UI, while it would be convenient to have a command that let you perform the same export without launching the whole UI.

Launching the whole Logseq and its UI on a server and simulate the human interactions look like a waste of resources and energy that at the moment is really important and we shouldn’t waste it.

Is there a reason for Logseq to have the need of a whole Web UI to work? I don’t think ClojureScript needs that, right?

A related question is: can’t Logseq work as a background daemon and, when launched, the UI just interfaces with that daemon? This could make it faster to launch Logseq (UI) since it seems what makes its startup slow is ClojureScript bootstrap (right?).

I suppose Logseq works like it does because there was the need to run it client-side when hosted on a server like logseq.com, right? If Roam did this it made sense for their business model, but I don’t think it does for Logseq when run locally.

Hey @cldwalker, thanks for responding in this thread :pray:

My usecase:

  • I have a blog which is a generated static site that loads quickly and has a very simple UI, I don’t want to replace the blog UI
  • I write my blog posts in logseq, copy-paste the logseq page to a markdown file, polish the syntax and publish the blog post
  • The moment I copy-paste the content, the original draft in Logseq is completely disconnected from the final product

GitHub - sawhney17/logseq-schrodinger: A plugin to export pages in Logseq to Hugo. is a partial solution, but it is a logseq plugin so I have to manually click to export public pages

It would be fantastic if logseq supported a workflow that can programmatically produce Markdown/HTML files for all your public pages.
It would be a CLI script that takes

  • path to a graph
  • path to a destination folder
  • and parameters that are taken by the “Export page” feature

And the CLI script then produces a flat list of files with exported public pages. In an ideal scenario, it could also generate a metadata JSON file where you could find page attributes for each page.

Edit: I just thought of an alternative: make it possible to install plugins and run their commands from the command line, then I could just use the GitHub - sawhney17/logseq-schrodinger: A plugin to export pages in Logseq to Hugo. or any other plugin to export the pages.

The key requirement is that you could run this from CI/command line. This would allow the same workflow as GitHub - pengx17/logseq-publish: Logseq Publish Action but without the need to use the full-fledged Logseq web UI

My current best option is to grep my graph/pages folder for public:: true and then do post-processing on the raw graph Markdown files

1 Like

Hi and thanks for the link to the GH Action.

Is there a roadmap maybe like Dynalist has on Trello, where people can vote on features?

Here are a couple of ideas/suggestions I had from a past post:

Re: command line, the two other replies had some great comments on why to use a more Unix like design philosophy. In short, interoperability is king, especially for utilities. You want Logseq to be able to be called by whatever means a user could dream of, and the most simple is from the command line. Well, APIs if it is webbased, but APIs are basically CLI commands over the web.

Cross-platform electron apps can still use CLI: https://binary-studio.com/2017/09/21/implementing-cli-electron-apps/ These days Golang is becoming the most popular cross-platform basis and it is very CLI friendly. Also, since so many users will turn their notes into blogs and websites, and Linux is the backbone of 80% of servers in the world, the CLI is the first place to make this process more usable.

1 Like

A quick note… I found the Feature Request list in Discourse below and the number one request is for Vim shortcuts. Does this not speak to how important Unix is to users?

Using the Unix philosophy, one would not reinvent Vim shortcuts for one’s app feature, but simply allow use of Vim for editing and export the output to whatever medium a user wants… using pipes from the command line. I feel like the cart is before the horse here. I realize the app is amazing at providing a Roam like utility, but I’m coming from the other end of wanting to use org-mode and markdown (backlinks would be great) with Workflowy-like outlining. So building from the bottom up of Unix solves so many current (and future) problems because they have been doing it longer than anyone else.

Hey everyone, I wrote a small command line program to export public pages from Logseq. The downside is that it is quite limited in the page syntax that it handles. The upside is, that it’s a 2MB executable that can be run independently of Logseq (ideal for GitHub actions or GitLab CI/CD).

5 Likes

Thanks for sharing, it’s really useful also because it should produce .md files compatible with Quarto to produce dynamic scientific documents like articles, presentations, interactive websites and books.

GitHub - pengx17/logseq-publish: Logseq Publish Action

When I read how it runs, it’s super over complicated… to me it is not a commandline to run it, IIUC it is more like “automation to run logseq desktop and click export pages” instead of having actual programmatic triger.

Not mentioning , that instructions how to run locally are far beyond “easy adoption”. It should be oneliner in shell, to make it easy to setup e.g. as “git hook” Git - Git Hooks (e.g. to regenerate pages and automatically republish every time one commits to repository , for full automation and ergonomics)

1 Like

So there is this enhancement in the 0.9.2 release:

Make publishing accessible to CLIs #9033

Does it complete this FR (and if so, how can I make use of it?)
or is it related but different?

The publishing app can now be built with a CLI - GitHub - logseq/publish-spa: A github action and CLI to publish logseq graphs as a SPA app

1 Like