See
→ Movement on the Logseq Database version
Curious if there are already more infos out. Does this mean, future Versions of Logseq store file content and metadata in a local database like SQLite and not in Markdown files anymore?
I saw somewhere from a team member, stated that Logseq will eventually store data in database only, but you can export data as markdown files anytime you want.
This would be huge! If you compare Logseq performance with other data-base driven apps - Joplin as an example here using SQLite -, there currently is a big difference in favor of Joplin. Not yet speaking of other features like block timestamps facilitated by a database-driven approach.
Yes. As an alternative, IIRC there have been attempts in a Joplin plugin to create a virtual filesystem mapper, so you still can read notes, as if they were plain Markdown files in the filesystem - despite they are actually stored in a database. Which would be a good sweet spot between people needing files and folks more driven by consistency, features and performance.
2 Likes
Below is just my personal humble opinion.
I’m an Obsidian user and I was introduced to Logseq just recently. Yesterday I found out about the database version and it was a disappointment for me.
I went to see a few videos about this DB version and my major fears were confirmed.
They started with adding meta information like fields, that can be attached to items. Which brings types and a whole set of problems that a regular person just cannot handle: types incompatibility, types conversion, old entities maintaining, data migrations.
What about references? Now you can reference just anything, but in the database version we can have a field of that type, like a reference to a page. Which looks sweet but… prevents us from changing our mind in the future. Will we able to change that too? How it’s gonna reflect on the rest of our data?
To address all problems that brings inventing fields, one needs to build a very complex system. The team can be spending years on this. And the more they add, the more users will request. There are no objectively limits when to stop. You will find yourself building not a PKM-system, but an application prototyping tool instead.
I remember CCK fields in Drupal. It’s one of the most flexible ui-driven user-defined fields building system I even seen. It took years and years of thousands of people to keep system shining. And yet, it was incredibly difficult for professional programmers to support and use it.
I realize it’s not necessary to go into all the complexities right away. But you will be asked to expose this in the API to allow for user plugins. So complexity will come from that side first. You’re opening pandora’s box. It’s happening with Obsidian now, where people just can’t stop making plugins (and it’s considered a good thing!), but users get constantly disoriented and get pushed away from building a solid PKM-database towards making ugly poorly maintainable app similarities.
In the end, one needs to ask themselves: what I want to build? An app prototyping tool or a good, stable and feature-rich PKM system?
–
The genius of this tool I discovered so far, was that it’s able to create ad-hoc blocks: blocks are all anonymous until you start referencing them, and then they acquire UUIDs! Bravo! And it’s done in a nice way, which is compatible with Markdown. It brings great flexibility! You don’t need to worry about WHERE you created block. You don’t need a think out a storage for it.
I mean this is something unique right now. It makes Logseq differ from Notion. This is great!
With SQLite database you don’t need to be witty. You can do just anything and it will never be enough as you’re basically not limited. It’s just that your focus is straying away from the essence of PKM…
Also, a show stopper for me personally will be inability to keep my data in my git-repos and to use git for syncing.
So guys, call me outdated, but I hope DB version will never happen.
Short answer
- The database version:
- will certainly happen
- It has been the top priority for months now.
- will have:
- compared to the current version:
- more features
- more stability
- almost none of the problems that you imagine
- compared to the current version:
- will certainly happen
Long answer
- To be fair, Logseq has:
- never been without major problems
- If in theory we lose the current version, we don’t really lose much.
- In practice, an open-source software cannot be lost.
- If in theory we lose the current version, we don’t really lose much.
- always used a database
- paid a price for not using it more
- A fully file-based stable and truly feature-rich PKM system is:
- almost impossible in contemporary file systems
- These are the ones that have failed to improve.
- much easier when dropping some features
- This is not a real solution.
- not worthy the troubles anyway
- There is no agreement on what a feature-rich PKM system is.
- Some users demand some features more than others.
- But there are great features that most users are not aware of.
- Most applications out there are really poor.
- No matter if some of their users fanatically praise them.
- We are still at an early phase of the software revolution.
- Even more in note-taking applications.
- Most applications out there are really poor.
- But there are great features that most users are not aware of.
- Some features have more demanding implementations than others.
- And there are big challenges that most users are not aware of.
- Some users demand some features more than others.
- almost impossible in contemporary file systems
- never been without major problems
- The database version:
- is an implementation detail
- a very big one, but still a detail
- An implementation detail is:
- capable of breaking things
- not capable of losing the important functionality
- doesn’t make Logseq a different application
- doesn’t change Logseq’s stated goals
- won’t be:
- the drama that some people fear
- Real trade-offs cause real friction.
- But time smooths-out most of it.
- the magic that some people dream
- The application will not do your work for you.
- the drama that some people fear
- is an implementation detail
- Here is what happens:
- People complain that Logseq doesn’t provide much information on the coming version.
- Then some info comes out.
- Then people make all kinds of assumptions.
- Humble or not, they are mostly wrong.
- Even worse, some people get frustrated or otherwise emotional about it.
- Then Logseq gets justified for not providing the info.
- To be clear, more information would be certainly welcome. But should be met:
- in good will
- Logseq may shoot their foot, but the chances for that are tiny.
- with an open mind
- There is more than meets the eye.
- and cold blood
- If too difficult, it is better to hibernate and get back when the database version is released.
- In that case, should actually stay away from beta software.
- If too difficult, it is better to hibernate and get back when the database version is released.
- in good will
2 Likes
The classic Markdown Logseq would be available still, as both options will be offered to your choice.
I think you have a good point here: an advantage of graph-oriented apps like Logseq is that you can place anything as a property value and it will be valid. So you can edit the Markdown files manually and still get valid results. But the DB version, for whatever reason, is forcing types after you set them. Instead defining types should have had been just a way to fill a menu to choose options, while still allowing to input anything manually.
1 Like
I think the database version is a mean to:
- address performance issues
- enable crdt/multiplayer
Property types can become a curse or a boon depending on how it’s implemented for sure.
For now, in the online db test version, it’s not possible to switch types yet, not sure if it’s coming or not… , but theorically, ‘string’ /‘text’ could be used as a fallback when user needs to switch types? (I believe that’s how nocodb, baserow or airtable work)
Maybe ‘text’ /‘long text’ could still be parsed for references just as in the the current file-based logseq version ?
A lot of the usability and power of logseq could be lost with the database version (like the ability to use external tools directly on files) but I also feel the perf issues with the current version are real and can’t be easily patched using the current architecture (this is strictly an uneducated assumption from me, but it seems both roam and logseq are not the fastest tools, maybe due to their underlying similarities).
Anyway I believe Tienson previously stated that both file-based and database-based logseq would still co-exist… .unlessvthey they changed plans ?
1 Like
Some weeks ago he confirmed the intention to make the DB version periodically sync with plain text files, so we may get the best of both worlds. Indeed there is no need to sync the files instantly like now and personally I would be OK with a Ctrl+S shortcut to save current page to its files if needed, plus allowing the user to handle the conflict if a file is updated on disk by third party tools and you try to overwrite it with Logseq’s Ctrl+S.
1 Like
Then it would be great to have those options.
Yet I believe it’s not gonna be an easy task to support both versions at the same time. Also, plugin developers will face this ambiguity.
We should also expect, that switching back and forth between DB/text version will not be an everyday task (or be possible at all), because DB version will obviously have more features that cannot be stored in text so one should be losing that information while switching to the text version. The reverse switch, however, should be fine.
A fully file-based stable and truly feature-rich PKM system is much easier when dropping some features
I like this approach. I hope that the new (database) medium won’t be a justification for a raising hell of features.
P.S. After 15 years of notes taking and especially - the last few years of Obsidian, I became kind of allergic to plugins and features. I see people enthusiastically build their complex databases and then get really hard times maintaining them.
No tool will ever have the exact set of features that someone would like.
- Would either:
- miss some features
- have some features in excess
- This is easily superior.
- Every feature is totally worthless, until the time that you need it.
- And every feature that you use every day, at some point in the past was totally worthless.
- Extension mechanisms is the right solution to this dilemma.
- But don’t blame the tool for its misuse.
- The real problem with features is not their number, but their low quality.
- Unmaintainable complexity is a form of low quality.
- In Logseq, one of the best features is the ability to disable some features: Settings > Features
- or rather keep everything disabled and only enable what you need
- Ideally, this list should be a dozen pages long.
Except timestamps, that would pollute too much the text files, I can’t think of any other information that can’t be stored in text files. And personally, timestamps are not worth the ability to edit text files.
I found a hackish way to “catch” all block properties, created by me or automatically by Logseq inside a Multi-Line Link Title (allowd by Commonmark):
Because Logseq adds each property just below the Block Header (first line/paragraph), if that first line ends with a [](. and the Block Body starts with a ') then it is a perfect markdown:
I had to resort to the [](. ') Markdown Link syntax in Logseq because Logseq doesn’t allow the MultiLine Title Syntax and I wanted it to look good in Logseq as well so I ended up using the complete syntax on each line:
example block [🏷️](. ')
SCHEDULED: <2024-03-30 Sat 11:27>
[](. ')
which messes a bit the Markdown Rendering in a Markdown-Capable App (ex: VSCode, etc) but I can live with that as long as I can have all my properties in there, instead of loosing them at export.
Below is a complete example in an external renderer:
… and here it is how it looks in Logseq:
PS: I have used the “tag” symbol in Logseq to be able to hover on an actual “Link” in a Markdown External Renderer, otherwise I could have used an empty, null-pointing Link (also perfectly “legal” according to markdown syntax) in Logseq and it would look pristine (with the caveat that in an external markdown renderer one would not be able to see the properties unless the actual markdown file is opened with a text editor or parsed by a script:
In conclusion, not only that all properties can be “included” with the text file but they can even be made to be rendered if that would be an actual interest.