Support subdirs for namespace hierarchy

Currently namespaces are implemented by encoding the namespace path with %2F separators (soon to be replaced by double-underscores) as in persona/projects/projectApersonal%2Fprojects%2FprojectA. Logseq goes through a lot of trouble to avoid using the normal filesystem hierarchy.
There have been proposals here to just switch to using the filesystem, with lots of good reasons (filename length, CLI tool friendliness, import/export).
The current implementation does not seem to improve on using a filesystem in any way I can see, and it makes usage more opaque to new users and harder to integrate with other tools.
In fact currently logseq does support reading an existing subdir hierarchy of files as namespaces, so if you manually construct personal/projects/projectA logseq sees that page as being in that namespace hierarchy. But there is no way within logseq to create such a hierarchy. It’s so close!

This feature request is to please add a setting to create namespace pages using the normal filesystem, rather than using the custom path encoding.

References:

100%. I will never do more than just isolated outlining with Logseq if Logseq doesn’t play nice with Obsidian, Foam, etc. Even orgmode respected the filesystem.

3 Likes

In theory you can move .md files manually everywhere, in subfolders like graph/subfolder or graph/pages/subfolder and Logseq acts as they where in graph/pages.

But from Logseq UI there is no way to interact with this structure, at the moment it’s just a way to self-organize .md files in your filesystem.

4 Likes

I am curious why you would want to use folders…?

I have used Obsidian for about a year almost and rather quickly decided to use the vault folder for all pages with a few exeptions like daily notes and template notes. What do you gain with folders other then a structure outside of Logseq/Obsidian?

The linking becomes ackward, in my eyes at least, combining folderpath/filename as a link (looks really ugly too).

If I want to group notes I use tags which gives me the added advantage that a note can be in different groups what is not possible when you use folders for that. Unless you also use tags but if you do … why add folders in the mix?

And in practice (1300 notes now) I never have the need to name 2 different notes the same. Although I think it could potentially be an issue theoretically.

Personal preference could be a reason I guess.

Let me know why you want folders please.

Regards,
JeRoen

For me it’s mostly about interoperability with other tools. CLI tools (grep, df, sed, pandoc etc.), Emacs, org-mode, filesystem browsers, and so many others.
I get that tags are a more flexible way to organize notes (and they work on block level too). But namespace hierarchy has value too. And the logseq page namespace is 100% isomorphic to the filesystem; so unless the devs saving the filesystem hierarchy for some other use I don’t see why not use it. Why build a hierarchical namespace in the app and ignore the beautiful fast filesystem underneath it? You might as well put it all in a database.
I grew up with hierarchical filesystems; they’re in my blood and in my fingers. I stopped putting all my files in one folder in the '80s. So yeah, I guess it just irks me that logseq doesn’t take advantage of this feature of all modern OSes, and builds it itself. Seems like wasted effort at best, and source of new bugs at worst (like now they’re going to switch from %2F to __ because %2F is “ugly” – wouldn’t an actual filesystem be more elegant?
Anyway, one of the devs responded that this is a core design decision and won’t be revisited. (See Proposal: Changing How Namespaces Function in Logseq - #22 by alex0 “Our vision is to separate namespace structure from the limited tree structure of file system.”) So I guess this is not going to be pursued. :frowning:

4 Likes

As for Feat: new file name escaping rules by cnrpman · Pull Request #6134 · logseq/logseq · GitHub (replacing %2F with __ as namespace separator), I would be so happy if they would just allow customizing that separator, and I would simply choose /. In other words use the filesystem as is. (Yes, I know it’s not that simple – filesystem watchers have to be configured for subdirs and probably other things I don’t know about. It’s just so tantalizingly close already.)

1 Like

So they are going to change how namespaces work, it’s OK for me.

What I would like is that from Logseq UI I can somehow see/control the folder structure, for example it could:

  • ask in which folder a new page will be created
  • move to a certain folder the pages that have certain properties
  • turn All Pages view into a full file manager where I can move pages and assets into folders
  • in queries let me filter pages and blocks according to the folder structure, like we do with properties::
4 Likes

I’ve got a plug-in coming up that aims to handle this. :slight_smile:

3 Likes

What does your plugin do?

the logseq page namespace is 100% isomorphic to the filesystem;

No, it isn’t. The root of each hierarchy level in Logseq is still a regular note whereas a folder in the file system is only a container for notes. The elegance and simplicity of Logseq’s namespacing is that everything is a note. IMHO this is absolutely superior to hierarchical file/folder structures. (I also grew up with hierarchical file systems – and I despise them vigorously)

4 Likes

You’re right about that – to support both projects and projects/projectA being notes, there would also have to be a projects.md file as well as the projects/ directory. I kind of assumed that would get auto-created when needed…
I think Plan 9 and some other experimental OSes had something beyond a standard FS; what have you seen that’s an improvement? Just curious! Certainly AWS and Google cloud file storage are something a bit different, but they still support path-based access.

The major improvement for me is the ability to use faceted classification systems. If I need a classification system at all, that is. In most cases it is an utter waste of time that just soothes some kind of OCD-leaning anxiety.

Mind you: having worked as an Information Architect for some years I can totally relate to the notion of constructing arcane taxonomies for one’s notes. :nerd_face: Alas in the end I found that as a consequence of that you always spend way too much time on filing information (the more notes you have the worse it gets) and far not enough time on productive thinking/writing.

Hence I’ve started to throw out most classifications in favour of strong connections (i. e. direct contextual links).

Here is what I currently still keep using:

  • key type categories: projects (actionable, finite with an outcome), areas (actionable, infinite), resources/references (not actionable), indices and MOCs (the more meaningful main substitute for file system folders).
  • subject categories (e. g. people, organisations, locations, media, cadences), all of them with some flat hierarchies, e.g. people/friends, people/family, people/authors, people/network, media/books, media/articles, loc/Berlin, daily/morning, …) – loosely informed by LATCH (⇒ UnLATCHed: Richard Saul Wurman’s Theory of Limitations | Isis Information Services)
  • process states (mainly for projects, i. e. projects/options, projects/later, projects/next, projects/WIP, projects/evaluate, projects/completed)
  • GTD contexts (with a bit of hierarchy, e. g. @private/home/kitchen, @work/meetings/1on1, …)
  • action and content tags (e. g. #decision, #idea, #observation, #insight, #belief, #assumption, #write, #follow-up, #expand, #prune, #clarify, …)

All namespaces as flat/simple as possible and never deeper than 4 levels. Most tags within a hierarchy are aliased to their last component in order to keep my notes free from clutter and speed up writing ( e. g. @private/home/kitchen => @kitchen).

Basically the vast majority of classifications in my graph exists to help me organise/structure work (actions) rather than information (notes/writing/thinking). For the latter direct links are way more useful and productive. Takes a while to let go of some habits acquired via managing more structured data (literature references, whatever…) but it’s totally worth it. For me that is. YMMV. :slight_smile:

3 Likes

My feeling remains that with deep collections of evergreen notes, tree-based graph-traversal is going to be needed, and this is always going to involve classification, which will always involve arrays of facets at some level, even if only implicitly.

A well-crafted tree lets you use procedural, navigational memory to traverse your graph, instead of struggling to remember enough terms to craft a tight enough query to capture what you want and filter out what you don’t want.

People complain that items can only exist in one folder, vs. being tagged to many concepts. But the value of foldering is in what it hides (every other file on your system outside of this group) as much as in what it contains. It’s value is focus.

I don’t care if under the hood a query is being used to generate a tree I’ve crafted. In fact, that might be a Logseq-style way to give us the flexible hierarchies used by Dendron, which can be easily refactored, and thus reorganized, in ways library scientists could only dream of!

I’d love to see Logseq’s namespace hierarchies work like that.

5 Likes

Hi folks; I would like to keep this feature request on-topic: supporting filesystem subdirs to implement the namespace hierarchy. (And yes I know this feature request may be dead due to lack of developer interest.)

Discussion of the usefulness of namespaces and hierarchies and other classification systems is interesting, but IMHO belongs mostly elsewhere.

This is a key point from @boisjere however:

People complain that items can only exist in one folder, vs. being tagged to many concepts. But the value of foldering is in what it hides (every other file on your system outside of this group) as much as in what it contains. It’s value is focus.

So if you think a namespace tree is useful at the page level, I’m interested to hear why the developers choose to implement their own namespace tree on top of a flat single directory rather than using what the OS provides (which would enable greater interop with OS tools). Perhaps there are some clever future ideas that would use filesystem dirs for some other concept altogether?

POSSIBLE REASON ONE: POLYHIERARCHY

One good reason would be to enable polyhierarchy - so a child can have multiple parents. This might be easier to implement over a flat file structure.

You could define one document tree, for example “Neuroscience”, to capture a growing collection of notes on various neuroscientific topics in something like a standard academic topic space (anatomy, physiology, perception, cognition etc). There might be a specific Page on perception in monkeys that matters to you there.

Then you could define another tree, like “Evolution”, and the same Logseq Page about perception in monkeys could appear in that tree as well. The same note could appear in two hierarchies.

This might be much easier to implement over files in a single directory than in a directory tree.

If files were moved into a directory tree used by the OS, they would probably be in a folder structure on the hard drive that matches the nesting order of the first hierarchy you define. The database would have to overlay additional hierarchies over those dispersed notes.

Maybe the database functions to allow polyhierarchy are less complicated if executed over a flat collection of files. That would be a good reason to just have a single Pages folder.

But I’m not sure this is actually the case. The database would have to map out which files are relevant to a hierarchy anyway.

I’ve been involved in discussing the idea of polyhierarchy in a few threads in the forum. It’s a rabbit hole, but may be of interest.


POSSIBLE REASON 2: REFACTORING

Another good reason would be to allow flexible hierarchy refactoring, like Dendron allows.

  • What Hierarchy Refactoring Is

Hierarchy refactoring is a “Find and Replace” operation that works over the filenames (and hence namespace positions) of notes. In Logseq terminology, it would work over Page names, and thus transform Namespaces/Hierarchies.

This is a massive innovation in knowledge organization, I think - on par with the block and bidirectional link in terms of importance. A potential revolution in knowledge management.

  • A Refactoring Example

Using my two topics of Neuroscience and Evolution, you could decide you want to situate all your biology topics within your Evolution hierarchy. Perhaps you’ve changed your mental schema for how all those things fit together. You now see Biology as a special case of a process that includes chemical evolution and computerized evolution simulations, and you want your namespacing to reflect your new organization of that knowledge.

  • How it Works in Dendron

In Dendron’s syntax, you could run a query for (Neuroscience)\.(.*), which would return all your notes (Pages) in your Neuroscience topic tree.

That’s the “Find” part of a “Find and Replace” over names in the Pages namespace for “Neuroscience”. Results of the “Find” operation are called the “capture group” for that query.

Then, for the “Replace” part, you could write something like (Evolution)\.(Biology)\.$1\(.*).

The “$1” symbol matches the first term of the capture group - so in this case it matches the term “Neuroscience”.

That would refactor your topic tree. So now if you traversed your graph from your Hierarchies root page, “Evolution” would be a top-level Name (like a folder), with disclosure triangles that would expose Biology, and within there, Neuroscience.

But then if you decide that organization of knowledge isn’t working for you anymore, you can refactor your Hierarchy again as easily as you Find and Replace text in a file.

  • Refactoring Redeems Hierarchy

A criticism of hierarchy as an organizational structure is that it’s both rigid and fragile. Dendron’s flexible and scalable hierarchies are neither.

Dendron is built as a VS Code plugin and is very onerous to learn for a non-techie. It’s aimed at technical users. Logseq supports hierarchy as an organizing principle, and is aimed at a broader market. If Logseq combined the best of Roam Research and Dendron into a single PKM tool, that would be extremely powerful.

It’s possible that flexible, scalable, refactor-able hierarchy is also more easily implemented by renaming files in a flat folder directory than by moving them around an OS directory tree. The representation of file location in the database might be a lot easier.

I’m not a developer, so I’m not sure. Maybe “move” operations are as easy as “rename” ones here… but my sense is that to support both polyhierarchy and flexibility, a flat collection of files would be a lot easier to handle.

2 Likes

Well, I don’t know about polyhierarchy; seems a bit complex to me (which hierarchy am I working with at any given point?) but I see it could be cool. And if that’s where the team is headed, then I guess a single filesystem hierarchy won’t cut it.
As far as refactoring (moving files in the hierarchy) goes, that seems like a huge win for a standard directory structure. Just grep for the filenames or content of interest, and move all those files into whatever subdir(s) you want. Either drag & drop in a file manager or just use find/grep/mv. Or python or whatever you like. Refactoring today basically requires logseq’s tooling, which is part of why it’s so frustrating to me. Why rebuild what the world has already made for you?

For me, it’s a question of wanting to use procedural memory and visual cues, vs. remembering concepts I want to surface. There’s a huge mnemonic value in just unfolding the tree that leads me to the “Shakespeare” namespace to add a note in a supra-page outline I can see, vs. being smart about linking, tagging, or looking up unlinked mentions, to tie it all together later.

In terms of “which hierarchy are you in”, basically in the “Hierarchies” section at the bottom of the page, I’d expect to see breadcrumb trails/outlines for each one. So I can navigate up or down whichever hierarchy I like. They’d all appear there.

The Page I’d just added to my Shakespeare collection of notes would be in my “Elizabethan Literature” hierarchy for making it “evergreen”, but it could also be in my growing “Dissertation Resources” tree.

I’m not sure how to use grep or python. I could drag and drop in a file manager, but I’d have to leave my PKM environment, which would break my flow a lot.

Right now I am super-frustrated by the lack of foldering or a better hierarchy function in Logseq too. So I dancing between using Logseq only, Logseq-plus-Obsidian over the same file base, or Obsidian-only. I hate this.

I need folders or hierarchies. I absolutely need to navigate spatially along a tree to get to the point in my graph where I want to let go and do free-form work. I’m a Context-First thinker. People who keep telling me I’m “doing it wrong” and should become a Random-Access or Daily-Notes-First (Time-based) thinker are not helpful. Their way is their way, with its own advantages and drawbacks, not a better way.

But I love the speed of outlining in Logseq. All I want is the same outline-based logic applied above the page level. If fast, flexible outlining of blocks ON pages is such a good thing, why are we suddenly denied the same cognitive manoeuver OVER pages? The Hierarchies function is right there begging for such a use.

I’m pretty much thinking I’ll need to use Obsidian as my main digital home, and my place for evergreen notes, and just use Logseq as a scratch pad.

It’s this folder thing.

I hate how clunky outlining and block referencing is in Obsidian, but those things slow me down, they don’t block a core cognitive function I have.

With Logseq, the problem is more serious. It’s fast and fun but it keeps wrecking my structures of thought by making the spatial organization of my ideas impossible to navigate.

But using both tools means leaving each one all the time, and futzing around with the file system too.

3 Likes

It’s an overly fancy word, but we are already using it intuitively on most e-commerce websites and library catalogs.

Here is an article how Polyhierarchies Improve Findability for Ambiguous IA Categories , the Nintendo Switch is part of both the “electronics” and the “video games” category.

The Amazon example on the Knowledge Management for Tags proposal shows how faceted searches can be based on this.

Breadcrumbs can show one or multiple hierarchies at the same time.

1 Like

This is the same point I thought about when thinking about a smooth hierarchical structure for pages.

If putting things in a hierarchical way is so good that it was chosen to be the essential way of showing and arranging items on a page, why it is suddenly an heresy to suggest its usage for pages?

OTHER POINTS

Reading through all the related posts in here, there are many good points arisen.

I’m wondering how many people are in our same condition: we genuinely think Logseq is a terrific app, but the absence of an easy-to-use, even just simple, way of hierarchically traverse just some of notes makes it too much difficult to use.

  1. My biggest fear is to forget the name of some pages and not being able to access them the moment I’d like to. This is frightening. (I don’t want to scroll through all the pages to find what I need. It won’t happen just once that I could forget the name of a page, over the long period it is tiresome).

  2. Another good point you made in a thread is about having one or more custom starting points to access (even just part of) the notes hierarchically, taking advantage of your memory. Recognition vs. Recall.

    2.1 This concept of ‘having a custom starting point” is also somewhat implemented with the current “Favorite” feature, which just pins pages on the left sidebar. So the concept seems to have been deemed worthy by the devs.

  3. The current Namespace feature is just a bit too cumbersome to use it in a smooth way.
    With files and folders you can just move things around, copy and paste them when you need to in just seconds. How do you accomplish that with Namespaces? You can’t move things around, you need to rename the namespace to ‘move’ a page into another namespace. Can you do that easily in seconds for dozens, or hundreds of pages?


From what it was suggested in all the related threads, the devs may think about implementing a way to (*of course not a comprehensive list, just brainstorming*):
  • Easily Create and Manage Polyhierarchies

  • Make a hierarchy favorite, so it pins like favorite pages

  • Possibly create a UI section dedicated to hierarchy (like they are doing with Whiteboards)




Another quick thought: we already have Polyhierarchies, but with pages being part of multiple blocks in different other pages.

3 Likes

With folders it’s easier to entirely archive done projects, it’s possible to combine notes and other resources into the same folder, those resources often need to be opened by other tools through the file system, not everything is about notes and attachments.

2 Likes