To improve zotero functionality in logseq, I have secured a small grant from an academic institution, logseq has generously offered to provide additional financial resources. Here is an overview of features, with links to relevant discussions and implementations, including in other open source markdown tools such as zettlr.
Here is hoping that just as we have done for the ongoing discourse graph implementation in logseq, where logseq is matching another grant I have secured from the same institution, we will be able to gather together a small group of devs willing to take this on, deliver key functionaliyt and a perspective on maintenance of such a key feature.
If you fee like you are the one(s) who can implement some of the key features belw - or if you have even better ideas for a local first, robust integration of logseq and zotero for academic use cases, psot here and/or message me. Looking forward to kickstarting better zoteroing in logseq and better loseqing with zotero!
Zotero and logseq integration features needed:
native zotero literature search bar that works locally, this can easily be implemented as described here:
CAYW is very easy to use, but it returns the id, and I am not sure how to create a zotero:// link back to this. The following command will open a citation picker and pull all available information (needs BetterBibtex installed).
better integration of zotero and logseq file stirage and access, user Nhan describes a workaround:
How I set up my Logseq x Zotero is to store all PDFs in the Logseq assets folder, and in the pdf annotation page (the hls page) in Logseq I delete the 1st block that has the metadata for the pdf file (including absolute path, file, and title).
Without the absolute path in the pdf annotation page (the hls page), Logseq will default to look for the pdf in the asset folder that has the same name as the hls file.
integration of zotero and logseq files in ways that enable seamless cross referencing and access to logseq from zotero, this is partly implemented in this zotero plugin, where preliminary logseq suppport hs recently been added
icon transferral - use zotero item type icons in logseq - this sounds trivial but is a huge usability boon for every one who uses zotero as the icons are a key aspect of differentiating sources visually, eg book vs book chapter vs podcast vs video vs website etc)
PDF annotation transferral or sync: use zotfile-like abilities to either
pull pdf color-makrings made in zotero into logseq and/or
have a single file base where both zotero and logseq made pdf markings are visible from both apps
- better metatdata handling on import from zotero: allow selection or choice of metadata such as tags, the creation of author pages etc to be pre-set or adjusted on a case by case basis, as the current import of zotero items in logseq creates an abundance of new pages with varying usefullness. a well-structured source page and import filter settings would be preferrable
bells and whistles:
note template push:
start a book note in logseq , with a template, and continue working on it in zotero and vice versa
pdf markup integration:
have option in zotero to create logseq reference from pdf
have a way to have pdf markup interoperate between Zotero and logseq, possibly via a zotfile-like solution pdf markup extraction and integration
have a special graph that visualises item relationships as they are rdefined in the zotero database via the “related” info or through citations in item notes
To be clear: Imho, there is merit in keeping the functionality separate to the extent that zotero is a reference manager and logseq a knowledge management system. Of course here are overlaps but there is no need to replicate zotero functionality in logseq if the integration of zotero in logseq is well done.
I think the technically superior approach would be to approach the Zotero devs and ask for a local API exposing most of the functionality over a local web interface. Then Logseq could easily fetch the currently selected item, search, or add links back to any Logseq page citing an item. In principle this capability mostly exists through the debug bridge, but this is not officially supported, so I would not rely on it.
There is currently an ongoing discussion about integrating a local endpoint.
Less coupling and less features (no back annotation into Zotero) via automatic BetterBibtex export. Maybe the owner would be willing to also dump the zotero item-ids and attachment ids into the .bib file. With this information, Logseq could easily link back to Zotero with zotero://select links, but one couldn’t add links to Zotero linking back to the referenced pages.
I think you and your team really need to be aware that both Logseq and Zotero will keep on updating, that the Logseq PDF engine will update and that one day Zotero 7 will come. Creating a comprehensive plugin that addresses everything that you mention might become extremely cumbersome to maintain.
Therefore, and as you said, try to keep the functionalities separate as much as possible and think what should be ideally done in Zotero and what in Logseq. With the PDF viewer in Logseq, you should i.e. assume that people will not be taking notes in Zotero, so do you really need a way to seamlessly import/export your notes and vice versa or to have the higlights synced? Sure it’s a nice to have, but not a priority since it means more work and potential breakdown risk when compared to the other features.
Your goal should be on how to make the Zotero imports in Logseq a well managed workflow in Logseq, not across platforms.
Probably you’re already aware of it, but a feature I didn’t see in your list is a way to customise the import template with Zotero info.
@andupaz great comments. I am just giving ideas here, not making any decisions, but I absolutely agree with your comments. From my point of view, this would be the ideal end result of the integration:
Logseq has a way to cite Zotero items
Cite by searching for name etc.
Cite selected items in Zotero
go to Zotero window, select all the items desired, then run command
this is useful to cite e.g. a bunch of related items in Zotero
Cite last n items added to Zotero
this will speed up things a lot, if I add 3 articles to Zotero via the connector, I can immediately add them to Logseq (where probably most of the work will be done), without doing a 2nd search.
Once the articles have been added to Logseq, their functioning is independent of Zotero
The only coupling back into the Zotero-world is via zotero://select links
pdf files are directly linked (i.e. not via a webserver through Zotero)
This means, if Zotero is removed at this point, everything will continue to work
If the local Zotero-api ever stops working, we can switch the citing mechanism to a .bib file based system
The link Zotero → Logseq is done by adding linked items to Zotero
Every time an item is cited in Logseq, it calls the Zotero to add a linked attachment to the markdown file
If Zotero goes down, this feature will fail, but it is no big deal. There could be a routine to create the links after the fact once the Zotero issues have been fixed.
This plugin allows you to integrate any citation manager that supports sync with BibTex including Paperpile and Zotero.
It’s been tested with 6k+ references and is also 100% local!
This plugin, currently supports inserting references by either creating a literature note page and linking to it (with fully user customisable templates!) or inserting a literature note inline based on a use customizable template.
This plugin is based off of the popular Obsidian plugin Citations
I think this addresses some of the major notes in this post. Let me know what you think!
Hi @Aryan. Thank you for the plugin! This is already much more useful than the cloud version!
I’ve a fairly big library, so the search takes a while, during which the window is non-responsive (no cursor echo)
I’ve done some quick tests:
For my library of about 10k references, parsing the file takes a couple of minutes,
without smart search, it takes ~15s
with smart search, ~40s.
smart search, index abstract disabled, ~10s
The bibtex export takes a couple minutes. Despite checking the box “Keep Updated” it didn’t add any new items to the file for at least a full day (has nothing to do with your Citations plugin, I’ll have to check if something is wrong with my Zotero install, but it is a serious issue)
BBT exports the relative links to Zotero attachments, could you add a link to the pdf document for annotation in Logseq if the bibtex entry has a file = {…} item?
This is a bit of a hack, but a link of the form zotero://select/library/items/ABCDEFGH will point back to the item in Zotero. The identifier ABCDEFGH is also the name of the storage folder. So if the item has a file entry, you can parse that and use it to add a link back to the Zotero item.
Zotero also accepts links of the form zotero://open-pdf/library/items/9WYPFUSR?page=1 to directly open a pdf
I read somewhere that it was planned to add an option to open the parent item, not sure if that has been implemented.
Just to clarify, the time in seconds is the time for search to return results?
I think this is due to the fact that the plugin is rendering about 10k references in the background for each search query. I can probably reduce this so that it only shows the first 20 or so relevant matches and you can filter further using search.
I’ll look into the files, I’ve heard some feedback about this already and will be implementing it today or tomorrow.
Sorry, I’m not sure what you mean by opening the parent item.
@gax yes, that’s more or less it! @Aryan citation manager has a neat dropdown list which is still not completely fleshed out in the Zotero extension - thus a search function for Zotero items only would be definitely ideal (by author or title or year).
@Aryan I’ve been playing around with your plugin and it’s a breath of fresh air to be able to customise the template and have a dropdown menu. However, one major issue and one minor:
Major issue: What help is a citation manager in Logseq if there’s no way to open the PDF from the respective lit note? The way I see it is that it’s essentially the same as the Zotero extension, but without the PDF link and with customisable templates. Solves one problem, creates another. Without the PDF link, you’ll end up with two pages for the same document if you’re still using the Zotero extension to access the PDFs - creating kind of an unnecessary fuss. (especially if you pinned the bibtex citekey to your Zotero imports, with your plugin you’re creating two pages with the same name even). That just hurts the workflow and management of your bibliography.
Minor issue: The dropdown list is not searchable by author - normally in research you search by author names instead of titles.
Generally, I’d recommend creating and inserting instructional GIFs in the plugins Github page, so users can visualise the workflow. I had to look up a video of the Obsidian plugin in YouTube to know what good this plugin will do to me. Maybe mentioning no PDF links is created would also be good. I prefer having a non customisable template than no PDF link. If the plugin would have the link (and the minor issue mentioned above inserted) , the Zotero extension would be renderd more or less useless
Hope this helps for thoughts on future development. Cheers
@Aryan That’s great to hear! You already have an idea for when it will be introduced? The way Logseq makes it is fine, haven’t had any problems with that. The only thing missing for me was some way to directly open the PDFs without going through the respective lit page. Doing research I normally work with the same papers for a certain time, so having a way to easily open the recently opened PDFs to continue reading would be a plus, but not a priority at all.
But what I can imagine that you could do more easily is to have the dropdown list of references open from a button on the toolbar (like the “Tags” plugin) in addition to writing the command - this is more natural than writing a command in an existing note to just “search and open refernce as page”. (the cherry on the top would be to have the open pdf button directly beside the dropdown item without opening the lit page, but I assume this being more difficult to make )
In all cases, thanks for the quick reply, looking forward to having the PDFs incorporated!
Edit: Actually, since you can toggle between the functions (insert inline, go to reference, link to reference) in the dropdown menu, you could add a function to “open PDF”. Adding a button for these commands/functions to the toolbar,. which can be pinned/unpinned according to taste, would solve what I described above. The workflow would then be:
Needing to open a literature note?: Click button “Open Literature Note” on toolbar which opens the dropdown list with selected “Go to reference” command
Needing to open PDF?: Click button “Open PDF” which opens dropdown menu with selected “Open PDF” command.
Insert inline and link to reference would not need a button since these are used within a note, a button for these wouln’t make sense.
Hi @Aryan, yes, times are about the minimum times in seconds. Sometimes it takes a lot more time, up to about a minute.
The select for the parent items was something I picked up on the forum, but i can’t find it anymore)
Zotero has zotero://select links to open items. (these can also be found with the Zutilo plugin “Copy select item links”)
The links have the form zotero://select/library/items/[itemKey]
Unfortunately, they need the itemKey, and BetterBibtex currently does not export that item key (maybe the BBT author would be willing to).
We have the keys for the attachments, though. So one can add a links that open the attachment in the Zotero pane and the pdf, but not the parent item itself.
[Open attachment in Zotero](zotero://select/library/items/[itemKey])
[Open pdf in Zotero](zotero://open-pdf/library/items/[itemKey])
@andupaz@Aryan I agree, a command “add inline link to recently added citations” or similar with a list of the last 20 or so citations would be very useful.
Most people will have vastly more references in the bib file than references cited in Logseq, so it might be useful to have a way to narrow down the search for references that are already in Logseq with some type of command for the search window to narrow things down, something like author:, title:, local:, with local (for lack of a better command) limiting the search to items that already have a page in Logseq.
Locally present citations could also get a different color in the search results list and go to the top.
This would help to avoid creating duplicate citations, e.g. if I have author “John Smith”, I might have many entries in Zotero for different editions, drafts, preprints etc., so if I search for John Smith, there is a high chance to accidentally cite the wrong one and create a duplicate page. This tends to happen a lot with the Office plugin.
you can disable that if you want to speed up performance. reindexing performance is an issue with the bib format. I’ll add support for CSL json as that doesn’t require indexing at all.
great! There is also some ongoing discussion on the Zotero forum about adding a local API. Should that happen, one can get circumvent the file (at the expense of being tied more closely to Zotero).
@gax Now that the citations plugin has a link to the the PDF, I personally don’t see a use for the Zotero integration anymore. I’m no coder so I don’t know what a local API would contribute? Or why a Zotero integration would still be important?
@both Do you know of any way to link the hls page of the PDF to the reference template?
@andupaz it would offer you two-way integration, you could open an item in Zotero and have links to all your Logseq pages that cite that item.
You could select a couple items in the Zotero UI (e.g. all items in a collection) and cite them in Logseq without searching for each one individually.
Also it would be faster, currently you have to export your items from Zotero, then reload the database. Takes around 15min for a 10k library (but json export might speed this up, but then I just tried and it failed). The export is provided by a Zotero add-on. It is very popular, but not in the core. It might not be supported forever.
It is still an open question how to deal with notes and pdf annotations in Zotero, to integrate those will also require direct interfacing.
