Support subdirs for namespace hierarchy

Yes!
Haven’t used Dendron, but they, you and me came to the same conclusion (and who knows how many others). I also thought about ‘committing’ one of your virtual hierarchies to the file system.

We should perhaps create a new Feature-Request specifically for Virtual Hierarchies. Possibly highlighting the optionality of it, some people really seem to dislike folders, both there are many that do like them.

So allowing it to be a separate and optional feature may be an optimal solution for everyone.

At the end, it seems really simple:

  • Toggle in the settings to switch “Virtual Hierarchies” on and off
  • Dedicated side bar section (like Flashcards, or All Pages, Graphview…)
  • Opening the section you could see a Finder-like window with your virtual folders, files and pages together
    • Possibility to put together PDFs, images and pages in related folders
    • Common operations like drag-n-drop files into folders (important for quickness of workflow)
  • Commit a Virtual Hierarchy to “Hard” real File System Hierarchy

This is just a scratch, a quick outline of a rather simple feature as it seems (simple may not be equal to easiness of implementation).

What do you think about creating a dedicated Feature-Request? People could vote and give their insights on it.

Something interesting to note is that:
By adding this optional Virtual Hierarchies feature, the Namespace feature remains untouched, it will work as it works now, no need to mix them somehow, or change how they work just because of the Virtual Hierarchies.

2 Likes

It’s too bad the concepts of “Treemaps” or “Topic Maps” already mean other things. Because the idea that you can have multiple maps over the same territory is one that can be made clear.

The idea of making “Treemaps” over sections of your graph makes sense to me. It may be more accessible than the concept of a “Virtual Hierarchy”, although I myself don’t mind that term.

The “Treemaps” verbiage is useful though because the word maps can become a verb. So you can “Treemap” a section of your graph. Treemapping can become an activity.

It also makes the practice seem more lightweight instead of name spacing actual pages, you can reference pages into a treemap that becomes easily disposable afterwards. So you can treemap notes into an outline for writing a certain paper, for example.

As you point out, this is a separate issue from hard-committing a folder hierarchy. You can, in fact, create a treemap now, by hand-embedding page links into a new outline. You could also give all treemap pages a property like “role:: treemap”, and find all your treemaps that way.

It’s not namespacing. The idea of “committing” one of the hierarchies to the hard drive is new though. Probably basing treemaps on user-defined properties would make this difficult.

1 Like

You know what, I wouldn’t care if they did implement it and call it with some Hexadecimal text name, I’d just be happy to be able to use the feature.

That’s actually what I’m doing right. There’s no other way at the moment. I’m also using a special prefix character for these kind of Folder Pages.
Not the best experience as you can’t drag-n-drop and stuff, but we get by.

Hard-committing could be done later probably, if we can at least have a MVP for an optional Virtual Folder feature that’d be great I guess.

Should we ping someone from the dev team and pick their brain on the matter?
Specifically asking for what they think about an optional feature, separate from Namespace and other features, to create “Virtual Folders” (or whatever the name will be).

Pinging the dev team or creating a feature request certainly wouldn’t be a problem. Not great for me today because I’m about to take apart and reorganize my whole office. Also, I’ve been in so many conversations about hierarchies, folders, nesting, tree-crawling and classification now, that I feel the need to look over all that stuff and clarify what exactly is being asked for.

There’s this sweet spot between hierarchy-based tools like Dendron and traditional foldering systems, and block-based outlining with backlinks like Roam Research. It’s a market niche or technological niche that hasn’t been exploited by anyone yet to my knowledge, and there’s at least some interest in it among early users of Logseq.

But like I said, I’ve been in so many conversations about this now that I need to step back and look them over. Nevertheless summarizing ideas we’ve been discussing in a new feature request would not hurt anything.

Sigh. I guess this turned into a giant thing – my guess is it’s unlikely the logseq devs will want to take on something highly experimental that would add lots of new dev work like polyhierarchies, as interesting as they might be. I wasn’t looking for something super fancy. All I was hoping for in this feature request was to use the filesystem for the already-existing namespaces, so logseq can work better with the rest of the world.

@Gary_O see this proposal:

And this reply by one of the devs:

Last time I checked Dendron was all about the xxx.yyy.zzz naming convention, to the dismay of some users, in what sense did they “commit to the file system” since then?

They haven’t. I think the two issues are separate. If you had virtual hierarchies or treemaps, you could enjoy the same kind of hierarchy refactoring that Dendron enables. This hierarchy refactoring makes hierarchies less rigid and fragile. That is a desirable innovation for any system making use of hierarchy.

In terms of implementation, Dendron is using filenames. This is beset with problems, but undoubtedly there must be some tradeoffs.

“Commit to the file system” might be a useful feature for Dendron too, I don’t know.

I do know that hierarchy refactoring is a powerful feature for any knowledge management system though, and Dendron offers that feature.

1 Like

@Gary_O. I’d like to look at it from another angle:

  • File system interaction is important for many reasons:
    • Keeping subprojects separate
    • Interaction with other programs
    • Version control
  • The file system’s single hierarchy is not powerful enough. Foldering is a dead end. It doesn’t work on the OS level for organization and it is not going to work for Logseq. If Logseq limits itself to folder-like namespaces, it will be eventually wiped out by a program that does not have this limitation.
  • The file system has all kinds of obscure constraints (path length, disallowed characters, max items per folder etc.), violating these might lead to data corruption and loss. First priority should always be the integrity of the data, but we don’t want to subject Logseq’s names to these constraints.
  • The devs already committed to not implementing hierarchies as folders, which is a sound decision in my opinion.
  • The file system is a useful additional degree of freedom, and it would be wasteful to use this to just mirror Logseq’s internal hierarchy.
    • file system location can be used to decide what files to sync, where to store files, what to delete, use multiple git repos etc.

I think there is a solution to all of these issues. The first items are obvious, but the last one is problematic:

  • Logseq should stop trying to own our data by restricting files to its own folder and disallowing relative paths.
  • Every OS folder should be a potential location for Logseq’s files.
    • If I start a new programming project in a separate folder, I should to be able to document the project in a subfolder of my program under version control, keeping attachments locally or wherever I want them.
  • In the absence of overriding properties in the .md files, Logseq should use the OS folder hierarchy (which it already does, but I am not sure this is officially supported).
  • If there are additional properties specifying parents (multiple), folders become one additional facet to access the data.
  • Every OS folder structure will be a Logseq hierarchy, but not every Logseq hierarchy will be a folder structure.
  • As the folder structure is less expressive and more restrictive than Logseq’s, folders can’t be the default
  • This is the problematic one: Could there be some option, plugin (?) to map individual hierarchies to the file system? This is not as easy as it sounds, because now all functions that create new pages need an option to create pages locally. If I am on a page “A” and create “A/B”, should B be in a subfolder of A automatically, or should it be in the main hierarchy? Should this be a per-file setting that makes all subitems into subfolders?

In the spirit of keeping things simple, I’d say the first step should be to make Logseq officially accept files anywhere on disk, even if it means manually creating them.

1 Like

At the moment we can create subfolders inside a graph folder but:

  1. A subfolder can’t contain a graph of its own, since /subfolder/logseq leads to conflicts with /logseq. This could be fixed by simply ignoring /logseq in subfolders.
  2. Files with the same name in different folders lead to the same conflicts. It’s really hard to decide how to handle this:
  • Should different files with the same name be different nodes or the same one in the graph?
  • Should different files with the same name be displayed both as the same page in Logseq or should they be different?
  • How to reference specific files in subfolders or parent folders instead of others with the same name?

Edit:

Another idea to make this simpler to handle is to use an extension like .lq for Logseq files and treat other .md files we have in our folders as assets. .lq files would have to respect the rule of “no files with the same name in different folders” while assets are referenced with relative paths. This doesn’t solve all issues though.

1 Like

Hey,

Regarding Folders and Virtual Hierarchies. Maybe the following information could be deemed as off-topic but it could help visualize the usage of “Hard Hierarchies”, i.e. actual folders in the file system Vs. “Virtual Hierarchies”, just virtual folders, a collection of pages.

There is a photo-manager app called Mylio that actually has this feature of Folders and Albums.
“Folders” are actual folders on the file system.
“Albums” are virtual collection of files (photos in this case) and other Sub-Albums.

Here’s a visual representation and explanation:

It seemed nice to relate Logseq possible folder and virtual folder management with Mylio Folders and Albums management.
At least, this gives a real and tangible representation of what the feature could be, and how it could help users to better manage their files/pages in Logseq.

Also because, it was written a lot on this topic, maybe someone who wants to grab the idea faster, can just see the blog post or some videos on how this other program portrays the feature.


At the end, I'm not sure I fully understand the motives of removing options for users. If many users would love a folder and/or virtual folder management *in addition* to the current link/backlink management, why just not give it to them?

What is the damage caused to users by adding this feature?
If there is no damage for user, then why not considering it? Just because “it breaks the current design”?
Is that a damage for users?

Struggling to understand.

If, on another side, the reason for not adding this feature is about development workload and prioritization, then, that’s a whole different matter. But users should be transparently notified on the underlying motives of the decision.

1 Like

@alex0 I haven’t fully thought this through, but another option could be to keep Logseq links “[[]]” as they are, referring to local storage with unique names, but make Logseq recursively follow file:// links to .md (or .lq) files and incorporate them into the graph.
The advantage would be that any files in any folders could be part of the graph, even for regular Markdown files that don’t follow Logseq’s extensions.
This way there would be two namespaces, one for [[]] and one using regular relative links.

I am not sure this is worth it.

There is another feature request in the pipeline on Cross graph referencing. Not clear how this will be implemented, but it might at least allow to have separate graphs for individual projects, as suggested by @memeplex.

1 Like

Yeah, this is exactly what I meant.

Another option is to use symlinks in Logseq folder that points to files in the rest of the file system. If I can make Logseq

  1. ignoring a folder for git versioning
  2. not complaining if I don’t sync it on other devices
  3. ignore conflicts if files have the same names in different folders
  4. ignoring .md files in a specific folder

I know that there is an option to ignore a folder, but it doesn’t work for me, I need to investigate.


Edit: no need to set anything, all the files in /assets are ignored, no matter if .md or something else. So one can just symlink anything on the disk to /assets.

But it seems Logseq doesn’t display links that it doesn’t handle (.png images, .pdf etc). I would like if files where open with system apps.

1 Like

Anyway, for what it’s worth, I just came across Remnote. Local-only graphs as Logseq, Easy to write Advanced Queries as well.

But, it also allows for Folders! They structure everything as a block, no difference between Pages or Block, Blocks can be just blocks, or pages, or Folders.
And, yes, Virtual Folders as well.

PDF note-taking feature like Logseq as well.

I think it has everything Logseq has at this moment. Tags, Block/Page Refs, Block/Page Embed, Journal, Flashcards, Multiple Graphs, Plugins…
But also some more features, advanced flashcards control, cloud-sync for online graphs…

It seems they had some bug issues with some recent update, yeah it could be concerning, but I don’t know, I have a feeling it will be settled.

Sorry to make this post about another tool, but I am searching for a tool that suits my needs, and I tried Logseq, I really like it, especially because it is very fast in everyday operations. But I realized, part of my graph really needs to be put inside organized hierarchies, aka folders. That’s it for me, can’t make it without them.

I believe there are other people who may be in my same situation, so I’m giving my opinions based on the time spent on looking for and testing tools that better fits my needs.

You can organize subfolders in your graph folder the way you like. You just need to be sure that files in different folders have not the same name:

graph/pages/subfolder/example.md
graph/example.md

The above would give an error because Logseq doesn’t know which one contains the block for the [[Example]] page.

We are talking about the file system in our hard drives here. Does RemNote solve the issues we are discussing here somehow?

Sorry, I think we are talking about different things.

As far as I know, those are file system folders. I need to work in Logseq with Folders. With Folders I mean just Folders containing Pages, and I need to work with them, to manage them, move them, whatever, within Logseq.

What difference does it make to open the Finder and create folders from the Finders and moving pages around when those changes are not applied/not visible/not workable with in Logseq?

I’m not exactly sure what your needs are, regarding one-to-one mapping of real folders inside the OS file system and Folders inside Logseq/Remnote, but for me, I just need to open and use the note-taking app and being able to create folders inside of it (mere collections of pages). I don’t need them to be real folders in the file system.

Answering your specific question, Remnote does have export in different options, Markdown included. But I think internally, the management is different because you don’t see plain .md files in the graph (Knwoledge Base in Remnote terms).

I’m working on a plug-in that should address your use case in particular @Prote

1 Like

Indeed, you are in the wrong thread, did you read the posts or only the title? :wink:

What you are looking for has been discussed here:

And here there is my proposal:

You would need the above plus a convenient UI to manage hierarchies like we do with folders in our drives.

@Prote would you be happy with a nicer UI (e.g. drag & drop like a file manager) on top of the existing “namespace” hierarchy, i.e. “folder1/folder2/page”? Or are you thinking something different than namespaces as implemented in logseq now?

That’s the kind of paradigm that makes sense to me and it’s what I"m personally working on