Support subdirs for namespace hierarchy

the logseq page namespace is 100% isomorphic to the filesystem;

No, it isn’t. The root of each hierarchy level in Logseq is still a regular note whereas a folder in the file system is only a container for notes. The elegance and simplicity of Logseq’s namespacing is that everything is a note. IMHO this is absolutely superior to hierarchical file/folder structures. (I also grew up with hierarchical file systems – and I despise them vigorously)

3 Likes

You’re right about that – to support both projects and projects/projectA being notes, there would also have to be a projects.md file as well as the projects/ directory. I kind of assumed that would get auto-created when needed…
I think Plan 9 and some other experimental OSes had something beyond a standard FS; what have you seen that’s an improvement? Just curious! Certainly AWS and Google cloud file storage are something a bit different, but they still support path-based access.

The major improvement for me is the ability to use faceted classification systems. If I need a classification system at all, that is. In most cases it is an utter waste of time that just soothes some kind of OCD-leaning anxiety.

Mind you: having worked as an Information Architect for some years I can totally relate to the notion of constructing arcane taxonomies for one’s notes. :nerd_face: Alas in the end I found that as a consequence of that you always spend way too much time on filing information (the more notes you have the worse it gets) and far not enough time on productive thinking/writing.

Hence I’ve started to throw out most classifications in favour of strong connections (i. e. direct contextual links).

Here is what I currently still keep using:

  • key type categories: projects (actionable, finite with an outcome), areas (actionable, infinite), resources/references (not actionable), indices and MOCs (the more meaningful main substitute for file system folders).
  • subject categories (e. g. people, organisations, locations, media, cadences), all of them with some flat hierarchies, e.g. people/friends, people/family, people/authors, people/network, media/books, media/articles, loc/Berlin, daily/morning, …) – loosely informed by LATCH (⇒ UnLATCHed: Richard Saul Wurman’s Theory of Limitations | Isis Information Services)
  • process states (mainly for projects, i. e. projects/options, projects/later, projects/next, projects/WIP, projects/evaluate, projects/completed)
  • GTD contexts (with a bit of hierarchy, e. g. @private/home/kitchen, @work/meetings/1on1, …)
  • action and content tags (e. g. #decision, #idea, #observation, #insight, #belief, #assumption, #write, #follow-up, #expand, #prune, #clarify, …)

All namespaces as flat/simple as possible and never deeper than 4 levels. Most tags within a hierarchy are aliased to their last component in order to keep my notes free from clutter and speed up writing ( e. g. @private/home/kitchen => @kitchen).

Basically the vast majority of classifications in my graph exists to help me organise/structure work (actions) rather than information (notes/writing/thinking). For the latter direct links are way more useful and productive. Takes a while to let go of some habits acquired via managing more structured data (literature references, whatever…) but it’s totally worth it. For me that is. YMMV. :slight_smile:

2 Likes

My feeling remains that with deep collections of evergreen notes, tree-based graph-traversal is going to be needed, and this is always going to involve classification, which will always involve arrays of facets at some level, even if only implicitly.

A well-crafted tree lets you use procedural, navigational memory to traverse your graph, instead of struggling to remember enough terms to craft a tight enough query to capture what you want and filter out what you don’t want.

People complain that items can only exist in one folder, vs. being tagged to many concepts. But the value of foldering is in what it hides (every other file on your system outside of this group) as much as in what it contains. It’s value is focus.

I don’t care if under the hood a query is being used to generate a tree I’ve crafted. In fact, that might be a Logseq-style way to give us the flexible hierarchies used by Dendron, which can be easily refactored, and thus reorganized, in ways library scientists could only dream of!

I’d love to see Logseq’s namespace hierarchies work like that.

2 Likes

Hi folks; I would like to keep this feature request on-topic: supporting filesystem subdirs to implement the namespace hierarchy. (And yes I know this feature request may be dead due to lack of developer interest.)

Discussion of the usefulness of namespaces and hierarchies and other classification systems is interesting, but IMHO belongs mostly elsewhere.

This is a key point from @boisjere however:

People complain that items can only exist in one folder, vs. being tagged to many concepts. But the value of foldering is in what it hides (every other file on your system outside of this group) as much as in what it contains. It’s value is focus.

So if you think a namespace tree is useful at the page level, I’m interested to hear why the developers choose to implement their own namespace tree on top of a flat single directory rather than using what the OS provides (which would enable greater interop with OS tools). Perhaps there are some clever future ideas that would use filesystem dirs for some other concept altogether?

POSSIBLE REASON ONE: POLYHIERARCHY

One good reason would be to enable polyhierarchy - so a child can have multiple parents. This might be easier to implement over a flat file structure.

You could define one document tree, for example “Neuroscience”, to capture a growing collection of notes on various neuroscientific topics in something like a standard academic topic space (anatomy, physiology, perception, cognition etc). There might be a specific Page on perception in monkeys that matters to you there.

Then you could define another tree, like “Evolution”, and the same Logseq Page about perception in monkeys could appear in that tree as well. The same note could appear in two hierarchies.

This might be much easier to implement over files in a single directory than in a directory tree.

If files were moved into a directory tree used by the OS, they would probably be in a folder structure on the hard drive that matches the nesting order of the first hierarchy you define. The database would have to overlay additional hierarchies over those dispersed notes.

Maybe the database functions to allow polyhierarchy are less complicated if executed over a flat collection of files. That would be a good reason to just have a single Pages folder.

But I’m not sure this is actually the case. The database would have to map out which files are relevant to a hierarchy anyway.

I’ve been involved in discussing the idea of polyhierarchy in a few threads in the forum. It’s a rabbit hole, but may be of interest.


POSSIBLE REASON 2: REFACTORING

Another good reason would be to allow flexible hierarchy refactoring, like Dendron allows.

  • What Hierarchy Refactoring Is

Hierarchy refactoring is a “Find and Replace” operation that works over the filenames (and hence namespace positions) of notes. In Logseq terminology, it would work over Page names, and thus transform Namespaces/Hierarchies.

This is a massive innovation in knowledge organization, I think - on par with the block and bidirectional link in terms of importance. A potential revolution in knowledge management.

  • A Refactoring Example

Using my two topics of Neuroscience and Evolution, you could decide you want to situate all your biology topics within your Evolution hierarchy. Perhaps you’ve changed your mental schema for how all those things fit together. You now see Biology as a special case of a process that includes chemical evolution and computerized evolution simulations, and you want your namespacing to reflect your new organization of that knowledge.

  • How it Works in Dendron

In Dendron’s syntax, you could run a query for (Neuroscience)\.(.*), which would return all your notes (Pages) in your Neuroscience topic tree.

That’s the “Find” part of a “Find and Replace” over names in the Pages namespace for “Neuroscience”. Results of the “Find” operation are called the “capture group” for that query.

Then, for the “Replace” part, you could write something like (Evolution)\.(Biology)\.$1\(.*).

The “$1” symbol matches the first term of the capture group - so in this case it matches the term “Neuroscience”.

That would refactor your topic tree. So now if you traversed your graph from your Hierarchies root page, “Evolution” would be a top-level Name (like a folder), with disclosure triangles that would expose Biology, and within there, Neuroscience.

But then if you decide that organization of knowledge isn’t working for you anymore, you can refactor your Hierarchy again as easily as you Find and Replace text in a file.

  • Refactoring Redeems Hierarchy

A criticism of hierarchy as an organizational structure is that it’s both rigid and fragile. Dendron’s flexible and scalable hierarchies are neither.

Dendron is built as a VS Code plugin and is very onerous to learn for a non-techie. It’s aimed at technical users. Logseq supports hierarchy as an organizing principle, and is aimed at a broader market. If Logseq combined the best of Roam Research and Dendron into a single PKM tool, that would be extremely powerful.

It’s possible that flexible, scalable, refactor-able hierarchy is also more easily implemented by renaming files in a flat folder directory than by moving them around an OS directory tree. The representation of file location in the database might be a lot easier.

I’m not a developer, so I’m not sure. Maybe “move” operations are as easy as “rename” ones here… but my sense is that to support both polyhierarchy and flexibility, a flat collection of files would be a lot easier to handle.

2 Likes

Well, I don’t know about polyhierarchy; seems a bit complex to me (which hierarchy am I working with at any given point?) but I see it could be cool. And if that’s where the team is headed, then I guess a single filesystem hierarchy won’t cut it.
As far as refactoring (moving files in the hierarchy) goes, that seems like a huge win for a standard directory structure. Just grep for the filenames or content of interest, and move all those files into whatever subdir(s) you want. Either drag & drop in a file manager or just use find/grep/mv. Or python or whatever you like. Refactoring today basically requires logseq’s tooling, which is part of why it’s so frustrating to me. Why rebuild what the world has already made for you?

For me, it’s a question of wanting to use procedural memory and visual cues, vs. remembering concepts I want to surface. There’s a huge mnemonic value in just unfolding the tree that leads me to the “Shakespeare” namespace to add a note in a supra-page outline I can see, vs. being smart about linking, tagging, or looking up unlinked mentions, to tie it all together later.

In terms of “which hierarchy are you in”, basically in the “Hierarchies” section at the bottom of the page, I’d expect to see breadcrumb trails/outlines for each one. So I can navigate up or down whichever hierarchy I like. They’d all appear there.

The Page I’d just added to my Shakespeare collection of notes would be in my “Elizabethan Literature” hierarchy for making it “evergreen”, but it could also be in my growing “Dissertation Resources” tree.

I’m not sure how to use grep or python. I could drag and drop in a file manager, but I’d have to leave my PKM environment, which would break my flow a lot.

Right now I am super-frustrated by the lack of foldering or a better hierarchy function in Logseq too. So I dancing between using Logseq only, Logseq-plus-Obsidian over the same file base, or Obsidian-only. I hate this.

I need folders or hierarchies. I absolutely need to navigate spatially along a tree to get to the point in my graph where I want to let go and do free-form work. I’m a Context-First thinker. People who keep telling me I’m “doing it wrong” and should become a Random-Access or Daily-Notes-First (Time-based) thinker are not helpful. Their way is their way, with its own advantages and drawbacks, not a better way.

But I love the speed of outlining in Logseq. All I want is the same outline-based logic applied above the page level. If fast, flexible outlining of blocks ON pages is such a good thing, why are we suddenly denied the same cognitive manoeuver OVER pages? The Hierarchies function is right there begging for such a use.

I’m pretty much thinking I’ll need to use Obsidian as my main digital home, and my place for evergreen notes, and just use Logseq as a scratch pad.

It’s this folder thing.

I hate how clunky outlining and block referencing is in Obsidian, but those things slow me down, they don’t block a core cognitive function I have.

With Logseq, the problem is more serious. It’s fast and fun but it keeps wrecking my structures of thought by making the spatial organization of my ideas impossible to navigate.

But using both tools means leaving each one all the time, and futzing around with the file system too.

1 Like

It’s an overly fancy word, but we are already using it intuitively on most e-commerce websites and library catalogs.

Here is an article how Polyhierarchies Improve Findability for Ambiguous IA Categories , the Nintendo Switch is part of both the “electronics” and the “video games” category.

The Amazon example on the Knowledge Management for Tags proposal shows how faceted searches can be based on this.

Breadcrumbs can show one or multiple hierarchies at the same time.

1 Like

This is the same point I thought about when thinking about a smooth hierarchical structure for pages.

If putting things in a hierarchical way is so good that it was chosen to be the essential way of showing and arranging items on a page, why it is suddenly an heresy to suggest its usage for pages?

OTHER POINTS

Reading through all the related posts in here, there are many good points arisen.

I’m wondering how many people are in our same condition: we genuinely think Logseq is a terrific app, but the absence of an easy-to-use, even just simple, way of hierarchically traverse just some of notes makes it too much difficult to use.

  1. My biggest fear is to forget the name of some pages and not being able to access them the moment I’d like to. This is frightening. (I don’t want to scroll through all the pages to find what I need. It won’t happen just once that I could forget the name of a page, over the long period it is tiresome).

  2. Another good point you made in a thread is about having one or more custom starting points to access (even just part of) the notes hierarchically, taking advantage of your memory. Recognition vs. Recall.

    2.1 This concept of ‘having a custom starting point” is also somewhat implemented with the current “Favorite” feature, which just pins pages on the left sidebar. So the concept seems to have been deemed worthy by the devs.

  3. The current Namespace feature is just a bit too cumbersome to use it in a smooth way.
    With files and folders you can just move things around, copy and paste them when you need to in just seconds. How do you accomplish that with Namespaces? You can’t move things around, you need to rename the namespace to ‘move’ a page into another namespace. Can you do that easily in seconds for dozens, or hundreds of pages?


From what it was suggested in all the related threads, the devs may think about implementing a way to (*of course not a comprehensive list, just brainstorming*):
  • Easily Create and Manage Polyhierarchies

  • Make a hierarchy favorite, so it pins like favorite pages

  • Possibly create a UI section dedicated to hierarchy (like they are doing with Whiteboards)
    WB Logseq UI




Another quick thought: we already have Polyhierarchies, but with pages being part of multiple blocks in different other pages.

2 Likes

With folders it’s easier to entirely archive done projects, it’s possible to combine notes and other resources into the same folder, those resources often need to be opened by other tools through the file system, not everything is about notes and attachments.

1 Like

+1 on the response of memeplex. I frequently need to store and retrieve files corresponding to projects.
Without file system hierarchy, I would always have to open LogSeq to find the right files, which makes a simple 5 second “select dir, select subdir, select file” task a 1 minute “open LogSeq, find the right note, find the asset link, find the asset in the asset directory”.
And should I ever decide to use another tool, I would have to go through all the assets and manually sort them into subdirectories.
I’m generally quite happy with LogSeq, but I don’t like feeling locked in.

I find it nice that there are ideas of using hierarchies in novel ways, but I would very much like to have the option to make the choice of having one of them being the primary one, and have it represented as file system hierarchy.

1 Like

You can always replicate the same hierarchical structure for notes and other stuff, perhaps using logseq namespaces for notes and the filesystem for everything else, in a way analogous to what Tiago Forte promotes in his PARA method. That said, there would be duplication of effort, archiving past projects as self-contained units would be almost impossible, etc. Given that logseq already works with files in the local filesystem it probably shouldn’t constraint itself to workflows popularized by cloud-based proprietary tools.

What about creating an optional feature for “Virtual Hierarchies”, like the flashcards feature, you can turn it on and off based on your likes.

Virtual Hierarchies will allow to create virtual folder and subfolders. You put your pages wherever you want. Meaning one file/page can be in multiple hierarchies.

Just like when you create a shortcut for in the Finder or Windows Explorer, but better because you will not see a difference between Original File and Shortcut, just your page being present in the folder(s) you have put it in.

This whole feature will be separated, you will have your graph as you have it now, but if you want you can activate Virtual Hierarchies and have a Finder-like view of your pages, with pages mixed with virtual folders and virtual subfolders.

Design of the core app functionality is untouched. You just have new views on your pages.
Also the problem of “if you have hierarchies it mean you assume your page can only be present in one folder” is fixed.

People who will not want to use it will leave the feature off, and people who want to use it will use it, and trust me, there are many people that would love to have a tiny ordered and hierarchical space for some of their notes, in addition to the beautiful linked and backlinked way of organizing pages in Logseq.

The people who would like a feature that will add hierarchies and not change the current design and philosophy of the app should come together and let the Dev Team know this is a really important aspect for many users.

1 Like

@Prote

You could achieve some remarkable flexibility by being able to construct “Virtual Hierarchies”, and then “commit” one of them to the file system. The hierarchy you commit would be the one you want to use for file management, directly in the OS directory tree itself.

The database would have to scramble around to re-analyze that “Base Hierarchy” (the actual on-disk folder structure) and map it to all the Virtual Hierarchies, if you change the Base - but an alert could pop up saying it can take time to Commit a New Base Hierarchy.

This would be something like Dendron’s flexible hierarchy paradigm, which enables many important innovations that Logseq developers may (eventually) want to consider.

@memeplex

With folders it’s easier to entirely archive done projects, it’s possible to combine notes and other resources into the same folder, those resources often need to be opened by other tools through the file system, not everything is about notes and attachments.

Both fixity and flexibility can be supported by having a Base Hierarchy you can Commit to the file system, and as many Trees (Virtual Hierarchies) as you like.

1 Like

Yes!
Haven’t used Dendron, but they, you and me came to the same conclusion (and who knows how many others). I also thought about ‘committing’ one of your virtual hierarchies to the file system.

We should perhaps create a new Feature-Request specifically for Virtual Hierarchies. Possibly highlighting the optionality of it, some people really seem to dislike folders, both there are many that do like them.

So allowing it to be a separate and optional feature may be an optimal solution for everyone.

At the end, it seems really simple:

  • Toggle in the settings to switch “Virtual Hierarchies” on and off
  • Dedicated side bar section (like Flashcards, or All Pages, Graphview…)
  • Opening the section you could see a Finder-like window with your virtual folders, files and pages together
    • Possibility to put together PDFs, images and pages in related folders
    • Common operations like drag-n-drop files into folders (important for quickness of workflow)
  • Commit a Virtual Hierarchy to “Hard” real File System Hierarchy

This is just a scratch, a quick outline of a rather simple feature as it seems (simple may not be equal to easiness of implementation).

What do you think about creating a dedicated Feature-Request? People could vote and give their insights on it.

Something interesting to note is that:
By adding this optional Virtual Hierarchies feature, the Namespace feature remains untouched, it will work as it works now, no need to mix them somehow, or change how they work just because of the Virtual Hierarchies.

2 Likes

It’s too bad the concepts of “Treemaps” or “Topic Maps” already mean other things. Because the idea that you can have multiple maps over the same territory is one that can be made clear.

The idea of making “Treemaps” over sections of your graph makes sense to me. It may be more accessible than the concept of a “Virtual Hierarchy”, although I myself don’t mind that term.

The “Treemaps” verbiage is useful though because the word maps can become a verb. So you can “Treemap” a section of your graph. Treemapping can become an activity.

It also makes the practice seem more lightweight instead of name spacing actual pages, you can reference pages into a treemap that becomes easily disposable afterwards. So you can treemap notes into an outline for writing a certain paper, for example.

As you point out, this is a separate issue from hard-committing a folder hierarchy. You can, in fact, create a treemap now, by hand-embedding page links into a new outline. You could also give all treemap pages a property like “role:: treemap”, and find all your treemaps that way.

It’s not namespacing. The idea of “committing” one of the hierarchies to the hard drive is new though. Probably basing treemaps on user-defined properties would make this difficult.

1 Like

You know what, I wouldn’t care if they did implement it and call it with some Hexadecimal text name, I’d just be happy to be able to use the feature.

That’s actually what I’m doing right. There’s no other way at the moment. I’m also using a special prefix character for these kind of Folder Pages.
Not the best experience as you can’t drag-n-drop and stuff, but we get by.

Hard-committing could be done later probably, if we can at least have a MVP for an optional Virtual Folder feature that’d be great I guess.

Should we ping someone from the dev team and pick their brain on the matter?
Specifically asking for what they think about an optional feature, separate from Namespace and other features, to create “Virtual Folders” (or whatever the name will be).

Pinging the dev team or creating a feature request certainly wouldn’t be a problem. Not great for me today because I’m about to take apart and reorganize my whole office. Also, I’ve been in so many conversations about hierarchies, folders, nesting, tree-crawling and classification now, that I feel the need to look over all that stuff and clarify what exactly is being asked for.

There’s this sweet spot between hierarchy-based tools like Dendron and traditional foldering systems, and block-based outlining with backlinks like Roam Research. It’s a market niche or technological niche that hasn’t been exploited by anyone yet to my knowledge, and there’s at least some interest in it among early users of Logseq.

But like I said, I’ve been in so many conversations about this now that I need to step back and look them over. Nevertheless summarizing ideas we’ve been discussing in a new feature request would not hurt anything.

Sigh. I guess this turned into a giant thing – my guess is it’s unlikely the logseq devs will want to take on something highly experimental that would add lots of new dev work like polyhierarchies, as interesting as they might be. I wasn’t looking for something super fancy. All I was hoping for in this feature request was to use the filesystem for the already-existing namespaces, so logseq can work better with the rest of the world.