Thank you for your thoughts and for your in-depth comparison!
You mentioned quite a few issues with doing annotations in Logseq that I wasn’t aware of. They are not unsolvable, so let’s hope that they will be fixed soon.
I’ve accumulated a lot of annotations in Zotero (using the old notes and now the new annotations), but it feels very limited. Having the ability to add block-level tags is quite nice.
I’ll need to have a closer look at the MarkdownDBConnect plugin, this type of plugin could solve the backlink issue for the loosely coupled approach via a bib file.
You are right about Zotero storage. I think it is also possible to directly sync the storage folder with syncthing or similar, just the database itself has to be sync’ed through the Zotero server.
I had a look at how the Zotero annotations are stored in the database: The annotations are stored individually in the sqlite file. Images are stored as regular Items in the storage folder. So most likely most users will be able to stay under the free tier if they sync the storage folder manually.
For me it is still not an option to upload all my database to the Zotero cloud due to privacy concerns, but it might be ok for some.
Personally, I’d like to move away from Zotero for anything beyond collecting and managing items. The architecture of Zotero is too closed for my taste. Moving items around is surprisingly difficult if not impossible, for example, moving items between libraries resets the created date, which would mess up my workflow. Also, Zotero’s tagging and filtering is lacking compared to Logseq, no hierarchies etc.
Hi there, a scientist is here. A heavy user of Zotero, Zettlr, etc. Very recently new Zotero plugin was announced, seems that the author is keeping it well updated. It is still not well known, but look promising for fast outlining and linking when working with PDF’s.
Thanks for interesting discussion!
I’d agree with the previous responses: a well thought out writeup of issues surrounding what is potentially a very useful workflow.
I’d hesitate to call myself a power-user in any of the programs under review (LibreOffice / LaTeX / zotero / logseq), despite a reasonable amount of experience in all.
For me, tight integration between zotero & logseq would be ideal. It strikes me that a useful avenue to pursue might be along the lines of zotero plugins for Libre(MS)office, which appear to reference local storage.
An equally workable solution would be for logseq to be able to import .bib files, much like LaTeX’ bibliography. This would obviate the need to work with large bibliographies.
For me, logseq’s ability to directly reference PDF’s in notes is a game changer
It might also be worthwhile asking what you require of each component of your workflow. I don’t require much more from logseq other than concept linkage & and export of a few dot points. I don’t require much more from zotero than to store references for searching. Any writing that needs to be done, I’m doing in the end program (LibreOffice or LaTeX, as the case may be) so that I can leverage the strengths of each component. However, it is useful to export a series of dot points with notes and references through (eg) pandoc (such as Zettlr) to the end program.
As far as integration goes, I found out that Zotero is not very open and that it is quite difficult to get access to the data locally.
I looked at the office integration a while ago, and it was very complex and limited protocol, that was also completely different between MS Word (COM-based, I think) and OO. There is also this protocol:
Overall, I am torn about the Zotero integration. I see that Zotero is developing quite slowly and I feel that relying on Zotero internals might be dangerous in the long run. My library has become very large, and the Zotero citation picker has become extremely slow, a problem shared by many users.
For some reason, Zotero does not provide a local API to access the database, so there is no official way to interact with a local Zotero instance (which is needed for privacy reasons and to work offline). Zotero also plans to switch to Electron, a switch which might or might not affect any plugins Logseq would rely on
For these reasons, I feel that the safest route is to go through .bib files (which would also open workflows with other reference managers).
An option for a tight integration could be to have a scanner that goes through the Logseq documents, finds any links to zotero, then opens Zotero and adds linked documents back to the markdown files. If the Zotero plugin goes down for whatever reason, it wouldn’t stop Logseq from working. I think this would be the best and most stable solution, short of an officially supported local API that exposes the full database (similar to Calibre’s API and the Content Server).
I agree with you that writing needs to be done in a word processor or LaTeX for the time being.
I’m new to Zotero so I don’t know much about it. Is it that you feel the development is slow or is this relative to another reference manager? Do you have an alternative in mind?
Could you give a few links of example of workflow using .bib file? I don’t know anything and would like to learn about this.
Zotero is a great program and I don’t see anything coming even remotely close, but still I have the feeling that Zotero is starting to lag behind. I am sure many problems are due to technical debt from being tied to the browser platform, this also makes it difficult to interface with 3rd party software. If you compare Zotero to Calibre, the latter has a much more vibrant developer company that has created a huge amount of plugins.
Over the years, I have run into many limitations of Zotero, such as
no easy way to transfer items between libraries while maintaining all information
no way to support complex workflows
search is very slow
too much emphasis on cloud sync, which has privacy issues
citation picker is very slow
no supported local API
tag system is primitive compared to how it should be.
no way to automatically populate collections based on tags (search folders have to hierarchy)
no automatic renaming of tags.
Zotero notes are great, but they lack Logseq’s features for assembling the information into other documents. Can’t tag individual blocks in Zotero’s Notes, tags are per note.
The new note support is great, but it still doesn’t support TeX, and currently there is no good way to export notes. Writing a note is a substantial investment (many hours per article), and I don’t like my notes to end up in a format that I can’t export properly. I don’t want to rely on a plugin either that might stop working in a few years when they move to Electron.
All of these issues could be addressed with a couple lines of Python, but the lack of a local API makes this difficult and one has to rely on the unofficial debug-bridge or write a Zotero plugin.
The Zotero development is also not very open, they have a mailing list, but no public roadmap.
I don’t want to be too critical of Zotero, like I said, it is a unique program, but I am still worried about putting too much of my intellectual work into the Zotero ecosystem.
There is a plugin for Better Bibtex that automatically writes a bib file and keeps it sync’ed. It still misses some information that would be useful (such as Zotero ID’s for zotero://select links, but probably the author would be willing to add those).
Logseq could then parse this file. This has some major advantages, it still works if Zotero is down and it doesn’t rely on the cloud, so no latency or privacy issues.
I wrote some more comments here.
That’s a good example for the lack of openness. Three years ago it was supposed to happen within half a year and now it has been postponed forever without much of an explanation. I don’t care about the GUI, but if the switch eventually happens it might break add-ons. I am also not very inclined to write add-ons for this reason.
I recommend a zotero plug-in called “Zotero IF pro max”. For highlighting content marked up by Zotero’s own PDF reader, it supports automatic generation and export of markdown files, with or without highlighting colors. The location of the exported file is the location of Logseq’s data. It is designed for Obsidian, but Logseq is also applicable.
The problem is that it’s a Chinese plugin, and that you have to pay for it. I’m not sure if it’s available in English. If you guys would like to try using translation software, I’m sure it would be very helpful. (Zotero IF Pro Max 首次使用须知
It works great! I have issues that Logseq doesn’t work with relative links (see Comprehensive Zotero Plugin - #42 by Luhmann ), but that is a Logseq bug.
It might be related due to me having the Zotero storage folder in a different location.
zotero-better-notes is great on this.
I take all my notes in zotero with zotero-better-notes, and then export markdown and sync them under Logseq folder.
Each note has a link to the reference pdf in zotero.
You can open the pdf from the note in Logseq with one click.
It work great.
I’ve tried zotero-better-notes, but the markdown file exported to logset is like, all of my annotations are all in one block. It’s kind of annoying I have to say
@yangjincai what export settings do you use from zotero-better-notes? (see snapshot below).
I screenshot an arbitrary selection, but I feel like whatever combination I try, the links aren’t working within LogSeq. But this is an amazing project, I hope I can get it working.
and if you want the [[bi-directional links]] work, you need to remove the random tag (avoid conflict) in export file names.
Zotero → Edit → Note Template Editor → ExportMDFileName:
Thank you @yangjincai !!! You just saved me untold hours trying to figure that out.
Also, the Zotero-better-notes plug-in sync is unidirectional…so accidentally overwriting notes seems over (especially over time when you forget about the syn and revisit a paper).
It’s frustrating because this solution is SO close to working if it could just sync both ways. Do any developers out there have a sense of how much work this would require to develop bi-directional sync? Like would it be an arm and leg to hire a freelancer or just a leg?
It’s quite frustrating, I must say, to see that the open source community is now, in 2022, finally able to start replicating the functionalities of a (relatively simple) commercial tool that had figured all of this workflow out about 10 years ago: Citavi. PDF highlights, annotation with page numbers that can be (1) tagged (2) categorized in multiple hierarchical trees, a plug-in that connects to word and shows this hierarchy in the text editor, automatic formatting of references…It’s all been there perfectly working and integrated (without the need to export, re-format, etc). Except of course it’s closed-source, becoming expensive, built on an outdated Microsoft database structure and is basically unable to move out of there (the web app is a joke).
If only Zotero and Logseq developpers could take a good hard look at Citavi and say: we can do that and much better, we would gain a huge amount of time. Anyway, in the meantime, we re-invent the wheel with the tools we have…For the time being, Citavi is such an efficient workflow that I’m willing to install a virtual machine and run Windows 11 just to be able to use that program. But I can’t wait until the day that I can switch to a good Zotero/Logseq integration…
I agree 100% - Logseq was my choice after trying most of the PKM apps around. However, I became incredibly disappointed to learn that there is no way to make notes on PDFs and then create a document where the references are automatically added (ie. parent block PDF citation).
Citavi is the model here. One can take notes, comments, quotes on PDFs and add them to Knowledge sections or chapters. And then in one click generate a Word document of these, with bibliography generated.
I highly recommend that Logseq put some resources toward solving this, via a plugin if needed. Even in the “Move Block” plugin, a left click for “Copy with ref” and then pasting giving the block with the reference as [[@reference_name]] would work find since we all probably prefer Latex anyway.
I was just describing a fast way to implement this, perhaps through the “Move Block” plugin. The main point is to have comments under a PDF link back to the PDF when inserted into a new note (presumably an outline of your paper).
Generally, we need to 1) Insert a PDF in any block, likely from Zotero and have a citekey 2) have any comments/blocks under that PDF include the citekey when added to a new note. Thus, using the Zettlekasten method we could quickly write new articles by opening a new TOC and adding relevant notes, and they would be “automatically” cited.
To be clear, this is just the fastest way and it could be built up a bit over time, by having preferences in citation type, format, etc.
