Would a rich commitment to hierarchies and classification be an anathema to Logseq culture?

First of all, I’d like to thank the adventurous team at Logseq for committing to the endless and sometimes thankless work of building a new software platform from scratch. I really appreciate what this tool is, and am excited for what it may yet become.

I’d also like to thank the company and key influencers in the Logseq community for not being entirely hostile to the idea of hierarchy in information organization. For many of us, hierarchy is a natural way to construct access paths into our graph.

I appreciate hierarchy as a way of clarifying thought. I don’t mind if it breaks down or goes stale from time to time. Cleaning it up, when I feel the need to, restores clarity in an important way.

I especially don’t mind if hierarchy is partial, in connected-thought environments like Logseq. I view hierarchies as lists of saved queries that provide me with a clean way into my graph. I appreciate them as a tool, alongside chronology (Journals), favourites, search, query, hyperlink traversal, graph examination, scanning the linked and unlinked references, and looking at tag clusters.

Hierarchies are filters over some of the graph that help me get close to 100% signal and 0% noise, at the cost of excluding interesting adjacent ideas. However, with a bunch of other access pathways in the mix, you can use hierarchy to find your starting point for a work session, then shift to other strategies of traversal. The Journals page is used the same way in many people’s workflows.

I wonder if other people in the community feel the same way about the value of hierarchy as one way into the graph?

If so, that would support the case for Logseq, and the culture surrounding Logseq, to differentiate itself from other tools in this space by validating hierarchy as a useful tool and an area for ongoing development. Not necessarily enforced, exhaustive hierarchy like a foldering system, but something between that and Nick Milo’s MOC concept. The distinction with MOCs would be in the level of core software support for hierarchy construction.

I personally would like to see a Hierarchies link in the left sidebar, right under the “Graph View” link. I created a sanitized little Hierarchies mockup. In this mockup, all namespaces start with the Hierarchies page reference. Then I used the self-referential embedding trick to grab all the child page links, and favourited the Hierarchies page so it lives in the sidebar.

This opens to show the following page:

If this looks suspiciously like a foldering system, it’s in part because it’s not an editable outline like any other Logseq page. If this was more like a typical page, and you could add bullets/content to it at each level, it would become more like an auto-generated master Map of Content for any pages pulled into namespace schemes. If you could add links to other related pages at will, it would be even more like an MOC hub, rather than a foldering system.

There are some interesting constraints on such a hierarchy page, including preserving outline level relationships more deeply into the graph, rather than just on the page. Cascading outline numbers down though the graph is also incredibly tedious right now - but that’s part of a whole other conversation on applying classifications over parts of your graph. I won’t poke that bear right now.

There are other conversations in this forum related to my hierarchies idea. There are feature requests for different ways of presenting hierarchies, showing breadcrumb trails in different ways, smoothing out how headers are opened or collapsed, or displaying a TOC tree for the current page in a sidebar, where twirling arrows down never opens up bullets that aren’t marked as headers, for cleaner structured access to parts of long pages.

It seems to me that there’s a single hierarchy-like function here that unifies these feature requests at some level of abstraction.

I could have made this topic into a feature request, but I wanted a more fullsome idea of what other people think first. Perhaps all this can be achieved with a plugin or something similar I have yet to discover.

  • Is hierarchy a valuable enough filtered view over part of the graph that it merits deep and careful development over a long period of time?
  • Is acceptance of hierarchy as an approach to knowledge organization a potential point of distinction from both the software and cultures surround similar TfT platforms (where hostility to hierarchy is widespread)?
  • Do you like and use hierarchies to keep your thoughts organized?
  • Would you like a system-generated-yet-editable Hierarchies view (of pages you’ve organized using namespaces) in the left panel?
  • I am missing absurdly obvious things in Logseq that allow me to achieve everything I want without needing to request a new feature?

After a long period of living in 'tool-choice hell", I am finding my home here with this tool, but I’m still really very new with it, so I don’t want to throw requests at the Logseq developers blindly.

7 Likes

Well I’m not sure about them potentially being “contrary to the culture” but I agree with you 100% that hierarchies and classification schemes are essential elements of an effective “knowledge repository”.

3 Likes

I personally consider hierarchies to be extremely important, if an item can appear in multiple parts of the tree. If a system would force a unique hierarchical classification of each item, and the item would only be visible at a leaf node, it would not be beneficial.

What I would like to see (some of this already works with nested tags):

  • Tags get a place in the hierarchy, e.g. “Historical Complexes” will be marked as part of “Non-Fiction”, “Non-Fiction” will be a part of “Writing Projects”
  • The relationship for tags and hierarchy positions has to be 1:n, i.e. each tag can have multiple parents or none. E.g. “Historical Complexes” can also have a 2nd parent “History”, which is a child of “General Interest”
  • Entry of tags should have some form of smart autocomplete, i.e. when I enter “History”, it suggests “Historical Complexes”
  • All items also appear under their parent categories, so when I click on “Writing Projects”, it will link to all pages under this category. The reason for this is that one should not have to go to a folder at the bottom of a hierarchy to find an item

As a generalization of the tag system, Logseq could implement nested searches. This would allow to place all pages (or blocks) that match a certain search into a folder hierarchy (e.g. “Historical Complexes” in the “Non-Fiction” search folder, which could also contain other subfolders. This would be an improved version of Zotero’s search folders, which don’t allow hierarchies. Such an approach is more powerful (e.g. one can have search folders that require multiple tags #history AND #fiction), but also slower.

3 Likes

The relationship for tags and hierarchy positions has to be 1:n, i.e. each tag can have multiple parents or none. E.g. “Historical Complexes” can also have a 2nd parent “History”, which is a child of “General Interest”

So I think you’re saying that Logseq should support polyhierarchies. That is when a child can have more than one parent.

So on a website, you could find an iPhone by clicking on “Electronics”, “Smartphones”, “iPhone”, to get to a target page, like “iPhone 7 Refurbished”. But up at the top level of the hierarchy with “Electronics” there could also be “Apple” and you could go from “Apple” to “iPhones” and get to the same target “iPhone 7 Refurbished” page by that route.

“Electronics” would be in a “by Department” facet array, and “Apple” would be in a “by Brand” facet array in this example.

All items also appear under their parent categories, so when I click on “Writing Projects”, it will link to all pages under this category. The reason for this is that one should not have to go to a folder at the bottom of a hierarchy to find an item

I’d like that too. That’s why I suggested a system-generated hierarchies page nonetheless be (almost) like an ordinary page, so you can be looking at parents and children to multiple levels of depth in one view. Is that what you mean? Maybe I’m not following you properly - or maybe I haven’t played with nested tags enough in Logseq. I did in Obsidian more.

As a generalization of the tag system, Logseq could implement nested searches.

Some of what you say makes me think again of the idea that a hierarchy is really a type of query that gives you a filtered view of a subset of your graph.

The underlying knowledge object is a graph with no top or bottom, but the nodes are constructed throughout outlining and indenting, which are mini hierarchies. It’s like the molecular/local structure is hierarchical even though the molar/overall structure isn’t.

Over time, the knowledge graph we use to store all our thoughts in loses any molar/overall hierarchical shape, but humans still use hierarchy and categorization to organize their thoughts at times. So picking a hierarchy as a lens for viewing as much of the graph as falls into its scope is a valuable view to offer.

1 Like

An example of a software that implements this beautifully is Trilium. It has many of the benefits offered by logseq. A disadvantage, and reason I started to look for another solution, is that it stores all information in a sqlite database instead of individual files.

1 Like

Yes, that is exactly what I was thinking about. Also, tags don’t need to be part of a hierarchy, so it doesn’t interfere with the current use of tags. On the other hand, a strictly hierarchical taxonomy, like e.g. the Dewey Decimal Classification would be a disaster, as many items could sit in many places of the tree.

There is also the W3C SKOS Simple Knowledge Organization System with a primer. This goes a bit beyond what we need here and also has a huge amount of metadata, such as notes about the individual categories, but the basic concept is to have skos:broader, skos:narrower, and skos:related to define relationships between categories.
These are not transitive:
broaderNonTransitive1
There are also transitive versions skos:broaderTransitive, skos:narrowerTransitive:
broaderTransitive1
There are also ways to specify categories for faceted searches.

I don’t know if SKOS is the state of the art and how this compares to OWL, but this is the general concept of what I would like to see implemented: a way to specify relations between tags to automatically build efficient searches.

I agree. Given a relationship between tags, many types of data presentations can sit on top of this, at the page level, block level, items expanded, not expanded etc. For example, in your “Hierarchies” view, there could be a button to fully expand the content, just show the first blocks etc. a faceted search could be placed in the sidebar.

I was thinking about hierarchical queries that can be used to build different hierarchical searches beyond tags. Currently, each query searches one level:

{:title [:h2 “Your query title”]
:query [:find (pull ?b [*])
:where …]
:inputs […]
:view (fn [query-result] [:div …]) ;; or :keyword from config.edn
:result-transform (fn [query-result] …) ;; or :keyword from config.edn
:collapsed? true
:rules […]}

So for example, to find a programming tag

#+BEGIN_QUERY
{:title “All pages have a programming tag”
:query [:find ?name
:in $ ?tag
:where
[?t :block/name ?tag]
[?p :block/tags ?t]
[?p :block/name ?name]]
:inputs [“programming”]
:view (fn [result]
[:div.flex.flex-col
(for [page result]
[:a {:href (str “#/page/” page)} (clojure.string/capitalize page)])])}
#+END_QUERY

If queries could be nested, then Logseq could automatically generate a hierarchy from the query. Each query would need a unique identifier and a list of identifiers of child queries:

{:title [:h2 “Your query title”]
:identifier “identifier of query”
:children […]
:query [:find (pull ?b [*])
:where …]
:inputs […]
:view (fn [query-result] [:div …]) ;; or :keyword from config.edn
:result-transform (fn [query-result] …) ;; or :keyword from config.edn
:collapsed? true
:rules […]}

When this query is run, it would present the search results in a hierarchical fashion. Of course, if one is not careful, this can lead to very complicated queries that might never terminate and it is far more complex than specifying relationships between tags. On the other hand, it would allow to build all kinds of interesting hierarchies beyond just tags, e.g. a hierarchy of all todo items which can then be refined by properties, or a list of all pages that have a pdf attached that can then be refined by tags.

I am not saying this should be the default way to generate hierarchies, relationships between tags is the way to start.

1 Like

Hierarchies should also be first class citizens of the Logseq world, i.e. it should be possible to search and return hierarchies in queries. We should be able to write queries like “show the results of the query starting at this point in the hierarchy as a hierarchy itself”.

Examples:

  • Show all TODO items in the hierarchy below tag T as a hierarchy
  • Search the hierarchy starting from tag T
  • Filter a hierarchy
  • Return search results as a hierarchy computed on the fly.
    • for example, search for TODO items and return a two-level hierarchy with e.g. priority levels first and people second.
  • Transform a hierarchy, e.g. flip people and priority levels in the example above

So I’m wondering how this would cash out in feature requests. It seems to me there are at least two.

I started this thread saying:

  • I’d like Hierarchies to be a system link in the left sidebar next to Graph View
  • I’d like it to go to a Hierarchies page that is editable, or semi-editable.

By semi-editable I mean the nesting order of system-provided links to nodes in the hierarchy would have to preserve their nesting order, but you could add sibling bullets to them to annotate what they are.

This would essentially turn them into a big Map Of Content - collapsible due to the outline format of Logseq. It would be built upon the namespace structure for pages, but pages can appear in more than one hierarchy.

That’s also compatible with what we might call “Tree Searches” - as in, return the tree to me; return the results in the hierarchy starting from this node as the full tree that branches out from there.

Hierarchies should also be first class citizens of the Logseq world, i.e. it should be possible to search and return hierarchies in queries. We should be able to write queries like “show the results of the query starting at this point in the hierarchy as a hierarchy itself”.

That is like a narrower case of my initial request, I think. My initial request could be to create a query that asks for “hierarchies; all”, and puts a permanent link to that in the left sidebar. The same underlying functionality could return “hierarchies; from-this-point-onwards”, to return a more focused tree.

Those are potentially both based on static namespace hierarchies alone.

Your other idea though, was more dynamic, if I’m not mistaken. It’s something like what Tinderbox can do with Agents, which are persistent save searches that monitor all input and trigger actions when something matching their search conditions appears.

So in this case, If I created namespace hierarchies that included “history” as a Zettelkasten topic tag, and “history” as a class of writing projects, then when I tagged a new note with “#history”, it would appear in tree searches for both Zettelkasten Notes and Writing Projects.

Is that right? In that case, that’s a kind of “autosort tagged notes into hierarchies” function that’s different from just displaying namespaces. But it could probably be made a superset of everything.

It might just happen that some users prefer to maintain tag discipline and very explicitly control the vocabulary they use to minimize autosorting. That would be a pain (I’d need z-history and w-history as tags to keep my Zettels separate from my Writing Projects) but a small one. That would give us cleaner hierarchies. People who want a more dynamic autosorting environment could use more general tags.

Am I in the right ballpark? I’m wondering how this all gets phrased as a feature request.

It looks like we are on the same page. We’ll need to split the inner logic of Logseq “what kind of structure the data contains” and the presentation “what kind of menu”.

For the feature requests, let’s start with a draft:

Feature request 1: Knowledge organization for tags

Rationale:

  • Putting tags into a hierarchy will remove clutter, improve organization, and enable more advanced searches, e.g. faceted and tree searches.
  • A single classification hierarchy (a la Dewey Decimal) is not enough, as each item can only have a single place (does a historic novel go under history or literature?). We want to find a historic novel to be both under the history and the literature section.
  • We need a polyhierarchy, where each item can appear in multiple places.

Implementation:
I suggest to roughly follow the SKOS system. Each tag can have optional relations specified that state that it is:

  • a broader or narrower version of another tag (e.g. “mammal” is a broader tag than “cat”)
  • a transitive version of broader and narrower (this probably should be the default)
  • related to another tag
  • identical to another tag (sometimes items might get tagged automatically, e.g. by Zotero’s imported tags, we might have an item with an automatic tag “History” and need to state that this is the same as “history”).

The other SKOS features, such as documentation, labels, and collections, are not essential and can be added on later.

Feature request 2: Faceted search / Hierarchy view

Rationale: Need an easy way to search hierarchies

  • Provide a hierarchies link in the sidebar that links to a hierarchies page
  • Provide faceted search (I am not a big fan of faceted searches and prefer a full tree view, but it has some advantages and can be naturally implemented on top of a knowledge hierarchy.)

Feature request 3: Tree search

Rationale: Hierarchies might grow very large, it should be possible to filter hierarchies to narrow down potential matches. The results should still be displayed hierarchically.

  • Searches can return trees, i.e. they can return a hierarchy. For example, a user has a hierarchy of books sorted by fiction/non-fiction, genre, author etc. Filter this hierarchy for all books that have an attached file will display the hierarchy with only the books that have a file attached.
2 Likes

Still mulling.

I think an actual faceted search is probably best implemented in a plugin. If the data model is there, then various kinds of facet search plugins can leverage it. It’s essentially a query GUI, isn’t it?

When I mentioned “facets” earlier in this conversation, I was actually referring to what gets covered off in SKOS as collections and concept schemes. Those can be used for analytico-synthetic post-coordinate classification like you find in Ranganathan’s Colon Classification or Bliss 2 classification. You can maintain a thesaurus, and implement some tag control that way.

I’m not sure that the idea of a single classification hierarchy like the Dewey Decimal system bothers me, because I don’t think classification in that sense is even possible in this type of environment.

The main issue with classification in a physical collection is that things need to only be in one place. In an index to that collection, or in a networked digital environment, they can appear in multiples places. An index isn’t polyhierarchical necessarily - it uses “see” and “see also” references to related concepts (skos:related) - but again that’s kind of vestigial from the fact that indexes have historically been created on paper. (One might say that “Unlinked References” is an evolution of these “related” index entries).

Feature Request 2: Hierarchy Page

I don’t know how hierarchies/namespaces are currently implemented under the hood. But my initial request, listed as Feature Request 2, is just the first bullet, more or less.

It’s agnostic to however namespaces are represented, and just asks for a quick link to an auto-generated list of them. It’s a lot like the “Hierarchy” link at the bottom of the page, except it lists all of them on a single page.

I can, in fact, hack this feature together now, more or less, by putting all pages that I’ve placed within hierarchical namespaces under a grandparent “Hierarchies” namespace, querying the namespace “Hierarchies” on the Hierarchies page, and favouriting it. It’s just not editable so I can’t make it into a “Master MOC” hub for my database.

I see this feature as a quick hit for the development team. It may not significantly derail their roadmap to squeeze it in. It would also be a clearly visible differentiator from Roam, if that’s important. For that reason, I’d pop it into the Feature Request forum as its own thing.

Feature Request 1: I Would Split it in Two

It seems to me that Feature Request can be unbundled a bit, split into two or three features. I’ll do it as two features.

FR 1.A: Polyhierarchies

Polyhierarchy is a distinctly different feature request. It could be implemented without there being an autocompiled list of all hierarchies, with a system link to it in the left sidebar (i.e. without Feature Request 2).

The “Hierarchy” section at the base of the page could simply be a “Hierarchies” section, and more than one hierarchy that intersects with that page could be listed there.

Now my ignorance is going to show here. Wouldn’t that mean that the place of a page in a hierarchy would have to be defined in a page property or page tag, as opposed to slashes in the title? I like deep trees and so the slashes in the title is a pain for me, so I’d be happy to do it differently.

I see references that indicate to me some other ways of defining namespaces in other places in this forums:

Anyhow, all that is to say that Polyhierarchies (having a page appear in multiple namespaces) can be implemented without having a sidebar link to an overview page, and vice versa.

I think it’s a different kind of feature request from Feature Reques 2, because it does touch how namespaces work, potentially.

I still think it’s a smaller feature request than implementing SKOS (provided Logseq isn’t architecturally similar to SKOS already).

FR 1.B: Vocabulary Control

Having a nested hierarchy of terms (broader, narrower, related) is different from having branching hierarchies of namespaces. In Ranganathan’s terms, it’s classification on the “conceptual plane” as opposed to the “physical plane” (I’ll treat digital notes as “things”, and so they’re analogous in some ways to objects - on the physical plane).

How are conceptual hierarchies different from hierarchies of things like notes? You can construct namespaces (for locating things like notes) any way you like. It’s essentially manual outlining, on a macro level above the page level. There are no conceptual limitations on what you put where.

However, once you get into the conceptual business of defining broader, narrower and related terms, you are doing vocabulary control. This is taxonomy creation. You are limiting what tags can mean, and how they are related to other tags. You can theoretically do this whether or not you have any instances of them yet.

You could import a taxonomy of animals, or a medical taxonomy, industries or minerals… whichever of those is important to you. There are lots of these official taxonomies, as you know. But this would give you a pre-structured set of properly-organized tags for organizing knowledge in the same way that your larger discourse community organizes it.

That would be very helpful for collaboration in certain contexts. I could see it as something a PhD student might do right away for getting some structure into their research database.

It would be cool to import a public taxonomy as a an outlined page, with indentation to represent each level in the nested hierarchy, but no square brackets around anything (or no hashtags). Then you could only turn the terms you want into pages, and potentially delete whole branches of the taxonomy that you are pretty sure will never relate to your work.

That kind of importer might not be core, at least at first. If structured research/discourse communities start to gravitate to Logseq, it might become a killer feature, but importing public taxonomies might be best left to a plugin for now - and it’s not what you’re proposing anyway.

It sound like you - like me - want to create your own bottom-up taxonomy of tags. I want that, with a thesaurus to keep track of (groom) term trees, to keep them clean.

I see Vocabulary Control as distinct from Polyhierarchy. Vocabulary control is very important to me as I envisage long-term use of these tools. I have no illusions that every note can or should fall within the scope of any one taxonomy. Taxonomies are MOC-style notes like any others, that structure segments of the graph - but they’d need to enforce the SKOS nesting relationships defined within them.

Just like on an MOC-type page for a manual namespace-based hierarchy, it would need to enforce the manual indenting levels you created across pages in the namespace - even though you’d need to be able to add sibling nodes at any level, to make the MOC more verbose and informative.

Idea: Logseq MOCs / Outliner MOCs / OMOCs

What I’m seeing here is the emergence of Logseq MOCs - or Outline MOCs (OMOCs) a distinct form of MOC native to outliners, with enforced indenting to preserve hierarchical relationships that span assemblies of notes (for manual page hierarchies) or ideas (for SKOS-light taxonomy-type conceptual relations) - but you can add notes to make them more informative than just TOC-like header links.

Question: Are Conceptual OMOCs Queries?

I think for the bottom-up conceptual OMOC, what I think you’re suggesting is the creation of a taxonomic index to notes. So you have a page with a taxonomy (you might zoom into specific blocks of it). There you find links to any notes you’ve decided to situate in that taxonomy, by tagging them with terms that fall within that taxonomy’s scope.

Navigating the taxonomy (the page defining the broader, narrower and related terms), would show links to the notes that have been gathered around those terms. It would pull notes into a hierarchy without using explicit namespaces. It’s a separation of the tree-making and leaf-placement concerns.

The concern of a taxonomy is hierarchy-construction. You make the tree as its own term-based thing, independent from notes, but it defines tags. You use those tags throughout your database. Then when you use the taxonomy page to explore the notes aggregated by that hierarchy, any notes you’ve tagged using those terms appear in place.

To express this second point a different way, notes flag themselves for inclusion within a hierarchy by wearing a tag belonging to it. (This is very Tinderboxy)

I see this taxonomic/conceptual plane organization as distinct from polyhierarchy. Polyhierarchy can possibly work with branching hierarchies of “things” only, not nested conceptual ones. So I think polyhierarchy and SKOS are different asks.

Splitting the First 2 Feature Requests into 3 Requests

I agree with you that if you are going to do any of this, it can theoretically be done using SKOS.

If I use namespaces to create branching hierarchies of notes, with no semantics that other people would care about, that can just be my own idiosyncratic instantiation of what, under the hood, is a concept hierarchy.

However, in terms of incremental steps towards developing these features, it’s possible that Feature Request 2, bullet 1, is quick and easy.

Polyhierarchy using the same namespace logic as the team currently uses might be a bit more work, but may not strain the existing architecture too much.

To allow the construction of SKOS-based conceptual taxonomies, and automatically aggregating links to notes tagged for inclusion in taxonomical indexes… the relative difficulty of doing this depends on how hospitable (or how “close”) the Logseq architecture is to SKOS already.

Perhaps its a North Star, and the team should try to make new developments future-compatible with SKOS, but it may not be something they can deliver in one incremental step yet. It may be more of a Saga than an Epic.

Feature Request 3: Tree Search

If you can aggregate leaf notes, so they appear at the right places in a conceptual tree for one big OMOC hub, which seems to be a query, to me, then Tree Search is probably a matter of adding a bit of tree-based syntax to cross-note queries.

I don’t know to what degree tree search query syntax is supported or achievable across multiple notes, in namespaces or so on.

Logseq is an outliner so there must be a ton of parent-child-sibling logic in there, within the page level. Presumably it could be elevated across pages to populate OMOCs. But I’m a newbie so I’m still figuring all this out.

Maybe this isn’t a feature request to the development team at all, but a user-community effort to figure out how to construct this kind of query. I don’t know. If it’s a feature request, it doesn’t seem as big to me as SKOS-based conceptual hierarchy support (enforced nested hierarchies).

Is All This Worth It?

It bothers me that when I express the desire for these library-science types of things, people often chime in suggesting it’s a high-investment, low-return activity with fragile output.

Many of these people understand that careful note creation is important, but suggest that structured outlining over a large collection of notes somehow isn’t.

I see a future where someone in their 20s today collects extensive notes for 40 years. Then they want to start writing masterworks. I think if they’ve been playing with and refining namespace OMOCs, and conceptual OMOCs all along, those will provide important pathways into their ginormous note clouds.

This isn’t foldering. These structured pathways into their graphs don’t need to be exhaustive. Not every note need be encompassed by these maps, and that’s fine.

I think you can often say about classification hierarchies what is sometimes said about plans. “Plans are useless, planning is priceless.”

When I spend time and expend intellectual effort organizing things hierarchically - essentially thinking like an outliner, but using crisp criteria over very large collections of notes, it increases my mental clarity. If at some point the hierarchy becomes useless, I’m happy to let it go stale.

When I’m writing a start with an outline but as I get into the compositional flow of things the outline gets very malleable and may be abandoned altogether as a new emerging logic asserts itself.

At the same time, a meta-outline that grows resilient over time, and seems to legitimately embody the larger structure of your way of thinking, would be gold.

tl;dr - I think we have 4 feature requests here, not 3. Actually… I think there are 5 - one is hidden.

  1. Namespace-based hierarchies homepage (the “Logseq namespace MOC”) autolinked from sidebar
  2. Polyhierarchy support (multiple “Logseq namespace MOCs” - where children can have two parents)
  3. Conceptual hierarchy (a complex query page that aggregates notes to the right nesting level in a conceptual tree on a page that has the property of being a conceptual hierarchy… ?)
  4. Tree-search - a query that picks a page in a multi-page hierarchy as parent, and returns it and all its child pages.

The hidden feature is enforcement. OMOCs would ideally be editable, with enforced limitations on indenting. OMOC indenting would have to enforce the multi-page nesting order of the multi-page hierarchy.

If you were doing an SKOS-like taxonomy to control vocabulary on a Conceptual OMOC, ideally that too would be editable. You want to be able to add sibling blocks for headings, comments, and maybe even manually link to additional pages. But if you’ve defined the hierarchy using either namespaces or concepts, the OMOC would have to preserve that nesting order. That’s an architectural saga, perhaps, and would be a fifth feature request.

Enforcing nesting levels like this doesn’t force page links to a single position in a hierarchy. If a page link is tagged with two concepts that both fall within the scope of some OMOC you’ve written, it would appear twice. It would have two parents. That’s fine.

Sorry for the rambling post. I can be very succinct - but I’m tired plus due to family stuff I step away and return frequently to this post. I haven’t had time to look back over stuff and redraft it during this time.

2 Likes

A much easier way to capture some of this is: Why can’t we indent lines on the “All pages” page?

Why can’t that lead to automatic namespacing? All root-level pages would just not be within a namespace.

Feature Request 2 could be a version of that.

1 Like

I fundamentally just really don’t like landing on the Daily Notes page. Nothing about my life prioritizes time as an organizing principle. It never has. My life patterns have always been irregular, and everything swirls around topics. I live in a world of topics. Time is a stranger to me.

I’m happy to have a Daily Notes page to use like a tiny piece of scrap paper off to the side, where I scribble down very, very minor things, but I never, ever want to open my software up and land there. It’s useless to me.

I never know what to write there because there’s no context to orient me. It’s not how I like to work. I know people say “just start typing”, like that should work with everybody.

It’s essentially like saying everyone in the world should navigate using path reckoning, instead of overview-based orienteering, in order to travel through space. I don’t work that way. I need to know where north, south, east and west are, and I need to orient myself in that great dome of space, to keep my position, direction and goal straight in my head.

You can tell me path-focused stuff like, “first go straight until you see the second Starbucks, then turn a slight left… and follow Gordon St to College Avenue”. If I don’t know roughly where those are in the great dome of space, I’ll still feel lost, and my travel will be inefficient. I need to know the overall spatial context.

Same with knowledge work. I like to begin from a point where I am well-oriented in my conceptual space. I really want better entrypoints, better framed in an informative context that orients me for work. So I began to think what kind of template would give me the access I need, but I didn’t get very far.

So I’ve hacked together a point of entry I like far better, based on hierarchies. When I open my software, I want to land here - with a panoramic view of all my most important entrypoints, so I can enter the graph at the right place.

But that damn hierarchy has bullets that won’t collapse to hide children. Plus the title of each page becomes absurd. If that info was displayed as a subtle breadcrumb line above the title, it would be awesome. As a page title it’s a disaster.

On that second screencap, the top part where I list my Journals and things, that is a completely manual set of namespaces I use to categorize segments of my graph.

But the bottom part, in the “Topics” hierarchy, that is where I’d want tools to do SKOS-like vocabulary formalization. I’d end up creating a classification scheme that fits my own way of thinking about topics - related to my projects.

I wish that simple {{namespace [[Hierarchies]]}} query I used to create this page generated a normal outline below it, instead of a static display of results.

This is just such a natural way for me to get to the right “diving board” to launch into a work session. It’s weird to me that makers of outliner software extol the value of outlining at the page level, to create parent, child and sibling relationships between blocks - but when you want to gain the same cognitive advantages of outlining over pages, you start running into resistance. It’s the same cognitive benefit, just applied at a higher level of organization!

4 Likes

Thank you for this excellent writeup.

I am not familiar with Logseq’s internals, but I think a plugin would be the way to start initially and let different ideas compete. Later on it might become integrated into the core.

I personally very much dislike these ancient classification systems, because I wasted a lot of my life trying to organize things in such a way. I see people on the Zotero and Calibre forums ask all of the time how they can maintain their library as a folder structure sorted by author/title etc, so these ideas are alive and still doing a lot of harm.

My worry is that, because everyone is familiar with library classifications, people might think that this is the state of the art and oppose far superior approaches.
I’ve received massive pushback at work trying to implement something that would have been trivial in a database, but management wanted and got a hierarchical folder structure that was virtually unsearchable and a complete failure.
I suspect that any proposal will get some pushback as well, saying that we can solve the issue with tags or with the rudimentary hierarchical page structure that already exists. This is why we have to explain very clearly how these old systems were made to deal with physical realities of index cards (à la Luhmann’s Zettelkasten) and items stored on shelves.

That’s why I like the SKOS concepts, they allow indices, (poly) hierarchies, and they can also be completely ignored and fall back on what we have now.

sounds good

I dislike specifying the full hierarchy for every item. That would also require memorizing the taxonomy or relying on autocomplete.
My thought was to have a separate taxonomy file that specifies the relationships between tags. In the case of taxonomies, these are very simple nested lists. In the case of general SKOS concepts it is more complex and would require writing down the relations, but this could also be mapped to Markdown.

The process would look like this:

  • specify relationships between tags through user-defined taxonomies.
    • for example, we have a document that specifies the taxonomy of animals. As this is a simple list that narrows transitively it can be represented by a nested Markdown lists.
    • Here is a small part of the classification of animals:
  • as you suggested, this list could be imported
  • adding a tag automatically places the item in the right spot of the hierarchy if the tag is unique in the taxonomy
    • A block on the common house cat will be tagged animals:F.catus, which will make it appear in the hierarchy in the right place, it will be returned in a search Felidae etc.
  • if the tag appears in multiple places in the taxonomy, a sufficiently long path needs to be specified to make it unique
    • I don’t think this happens much in good taxonomies, but it is no problem, if there where two species with the same name but in different phyla, one just has to specify enough of the path to make it unique.
  • if the tag appears in multiple taxonomies, we need to specify the namespace (probably this should be done for any taxonomy for future extensibility)
    • For example, the tag Felidae could point to the animal taxonomy, the novel, or the film with the same name. If a user has imported both a taxonomy of animals and of movies animals:Felidae and movies:Felidae make clear which one is meant (and an item might have both tags and appear in both hierarchies)

I am torn about this. On the one hand side, conceptual hierarchies, indices, related items etc. are different. On the other hand, they can all be represented by the same relations like SKOS does.
If Logseq had a way to specify the following relationships for tags:

  • taxonomies (as in the example for cats above, not strictly needed, but it simplifies things)
  • transitive and intransitive broader and narrower relationships
  • relatedness
  • (do we need anything else?)

then all of the concepts, from full imported taxonomies, user-defined taxonomies, indices could be represented using the same mechanism.
For this reason, I think from a software design point of view, they should all be integrated.

This is an excellent idea!

If the taxonomies were specified as simple .md files, it would be very easy to edit and share them.
Importing existing taxonomies would indeed be a killer feature for Logseq. What I especially like about this that it wouldn’t force the user to put any item into the taxonomy, like in a folder structure.
Let’s say I learn about medicine, if I start with a folder structure it will be mostly empty and it will be difficult to locate items. Also, as a beginner, I will probably not completely understand how things are classified in the field. On the other hand, if I can import the taxonomy, I can “tag” my items with one or multiple locations, or leave them as-is.
An item can also be part of multiple taxonomies, so if I work on the healing force of crystals, I can place an article on amethyst into both the medical and the mineralogical taxonomy.

Thanks for bringing up vocabulary grooming, the lack of which has become a major issue in Zotero. Items have user-defined and automatic tags (e.g. from a library import). I chose to use and keep the automatic tags (they are meaningful tags after all), which means that now I have thousands of items with tens of thousands of tags, many of which are duplicates, often just different capitalizations. If I deleted automatic tags, I would lose the classification of many of my items. Sadly, Zotero offers zero help in this, even if I rename items by hand, on the next import I might get unwanted tags back. Logseq absolutely needs a way to deal with this, as it will e.g. inherit the badly tagged Zotero databases.

they seem different from the point of the user, but under the hood they are all relations between tags. If we specify that TagA is a broader version of TagAA, we get a hierarchy (taxonomy), if we specify that TagA is a transitive narrower version of TagX and TagY we get a polyhierarchy that automatically includes all of the children of TagA (such as TagAA and TagAAA), and if we specify the intransitive narrowing we get only TagA as narrower versions of TagX. If we specify things as related, we get a “see-also” entry. If we want an index, we can rely on the ordered collections. So from a software engineering point of view, with just 5 relations, we can create almost any type of hierarchy.

I think this will come naturally. Once Logseq has a nice search hierarchy, people will want to filter the view as well.

I think it is absolutely worth it. I am fighting the massively lacking Zotero system of tags and saved searches every single day. I am at the point where it is often easier for me to find items on Google Scholar than in my Zotero database.
Logseq will only make this 10x worse. A researcher might add 5 Zotero items a day, 1000 per year; in Logseq, it is easy to add 50 blocks in the same amount of time. So we are talking about 10k blocks per year of research. Over the course of a PhD, we are talking about tens of thousands of blocks, over a career hundreds of thousands. Especially as there might be tools that automatically feed into the Zotero database, such as RSS-feeds and the like.

I completely agree. The main reason I am looking at Logseq is that Zotero just doesn’t cut it anymore to organize the information I’ve collected.

The benefit of these hierarchies is that they are so lightweight that ignoring them or adding new hierarchies is very little effort. This is not like a folder structure, which is fixed and can only be resorted with a massive effort. Here the hierarchies are a type of overlay on the existing structure, so despite the large power, it doesn’t complicate anything for users who chose not to rely on hierarchies.

So 1. is GUI representation, 2. is internal relationship between tags, 3. is different hierarchy views (?) and 4. is search on 1 and 4?

For the feature requests, we need to find the right balance between abstract terms and concrete examples.

While virtually everyone has used faceted search, e.g. on Amazon, only few people are familiar with the term, and abstract nonsense gets pushback.
We also need a good, concrete, example. Maybe something from literature. A book about World War 2 could be classified by

  • /Books/LastName/FirstName/Title
  • /Literature/Non-fiction/…
  • /History/20thCentury/WW2/…
  • /Military/Conflicts/WW2/…
  • /Places/Europe/…
  • /Technology/Weapons/…
  • Any of the Library classification schemes: Dewey, UDC, LCC, Bliss, …
    • These are available from library catalogs and a plugin can tag book items automatically
    • wouldn’t it be great to be able to browse our own libraries by these schemes?

All of which are completely reasonable and useful for finding the item.

We also need to explain why tagging is not sufficient, as in this case our item would need a large number of tags, forgetting to tag the item would make it irretrievable, and tags by themselves to not give the hierarchical search we need.

Here is an example why we need to be able to record relationships between tags.
This example is just about the tagging side, it does not say much about the presentation in Logseq, other than a browsable hierarchy as the most basic use case, but many more advanced searches can be built on top of SKOS-related tags.

Historian Bob studies animals in history.
He reads the book
Ark Royal : the Life on an Aircraft Carrier at War 1939-41.

Author: Sir William Jameson
Publisher: Penzance : Periscope Pub., 2004.

The book has these library classifications:

  • Dewey 940.545

  • LCC

    • World History And History Of Europe, Asia, Africa, Australia, New Zealand, Etc.
      • History (General)
      • World War II (1939-1945)
        • Naval operations
          • Anglo-German By engagement, ship, etc., A-Z

The book is tagged with Dewey:History of Europe and LCC:Anglo-German By engagement, ship, etc., A-Z. “Dewey:” is the namespace for the Dewey system, and “LCC” is the namespace for the Library of Congress Classification. These tags be added easily automatically by an improved Zotero plugin.
So just by automatically importing library classifications, we can already browse our books by the Dewey and LCC systems.

The book is also automatically classified by author last/first and year.

The book also mentions a cat Unsinkable Sam, so
Bob tags the book with animals:F. catus. This makes the book appear in a hierarchical search about animals as well.

So with very little effort (a single manually added tag so far), Logseq can already automatically generate 5 browsable hierarchies:

  • /Books/ByAuthor/Jameson/William
  • /Books/ByYear/2004
  • /Dewey/History and geography/History of Europe
  • /LCC/World History and …/History (General)/World War II … /Naval Operations/Anglo-German…
  • /animals/…/…/Mammalia/…/Felinae/…/F. catus

Additionally, Bob has a few lightweight classification schemes that fit his work, so he tags the book with bob:aircraft-carriers and bob:non-fiction, this additionally makes the book available under

  • /bob/military/navy/aircraft-carriers
  • /bob/literature/non-fiction/

Bob told Logseq that aircraft-carriers is narrower than navy which is narrower than military, so Logseq can also generate these search hierarchies automatically.

The process is very lightweight, so Bob can easily tag individual blocks of his notes.
It does not affect the current use of tags either, so Bob does not need to classify all tags from the beginning, he doesn’t even need to use the hierarchical capabilities at all.

The Library of Congress also uses SKOS, so one gets really nice search capabilities for free:

1 Like

I love this! I love the example, plus it’s more concrete, and as you say it offsets the abstract ideas.

This really makes the case for SKOS.

I wonder if we can prototype this by putting SKOS info in the page properties for this cluster of pages around Ark Royal.

In the parallel thread on hierarchies we see in the forums right now, “Using Hierarchies or Is There A Better Way”, they discuss how you can use page properties instead of namespaces to define a the parameters for a “trip” or travel page.

It’s @alex0 who gives the example. (Using Hierarchies or Is there a better Way? - #15 by alex0)

I don’t imagine there’s anything stopping us from placing SKOS fields in there. Later you might want a plugin to enforce proper sytax.

But for a prototype just for the cluster of BT, NT, RT relations around “Ark Royal”, we could probably hand-roll it, and then create some set of queries that spit out the 5 browseable hierarchies.

Yeah what you are looking for here is better UX for queries because once you embrance the “database” approach you can do everything mentioned here and more.

Logseq’s [[parent/child/teddy]] hierarchies are definetely not suited for this.

Better UX for hierarchies is certainly one of the feature requests we’re exploring here. You are right about that @alex0. It was my own animating concern once we got started, and I still want that.

Whether it’s built on top of namespaces, or properties and queries, I myself definitely want a “Hierarchies” system link, with qualities like I’ve described above, between the “Graph view” and “All pages”.

Also, in his outstanding work on SKOS polyhierarchies in this thread, I don’t think when @gax writes things like “/Books/ByAuthor/Jameson/William”, he’s necessarily literally referring to the current [[parent/child/teddy]] style of namespaces based on page titles.

As this thread has developed, @gax has contributed ideas that are growing so rich and powerful. I would just love to implement his most recent ideas in a small test case - just to click around and see what it would feel like to navigate a small set of pages through rich/smart polyhierarchy (his “Ark Royal” example).

I don’t care if it’s using namespaces, queries, or burning corn to appease the god Moloch. Really any mechanism for experiencing this type of page traversal in Logeq would be amazingly helpful.

I want to apply outline thinking at a higher level than “indented blocks on the page”. I want “indented pages in the hierarchy”. I want to apply exactly the same kind of thinking, one level up.

You misread my comment, I said “better UX for queries” not hierarchies and by that I mean just embrance the “database” approach with tags and properties and ask for a better UX to query them, so that you can browse them maybe interactively with “virtual” hierarchies.

For the use cases stated above I see the current Logseq data structure to be enough but missing representation/navigation.

For example [[Parent]] can have properties like

subcategories:: [[Child 1]], [[Child 2]]
another-hierarchy-subcategories:: [[Child A]], [[Child B]]
extends:: [[X]], [[Y]]
extended-by:: [[Z]]
generalizes:: [[W]]
whatever-you-want:: ...

but what is missed is UI/UX to easily browse them, for example a simple way to say “display the complete tree of pages formed by “extends::” property”.

Think this as databases vs file systems: databases are a more general data structure so you could have different “virtual” file system extracted from the same database.

Hi @boisjere and @alex0,

I’ve started to draft a feature request on:“Knowledge Management for tags / Hierarchical tags”. Please let me know what you think:

Why do we need Knowledge Management for Tags?

  • Tagging leads to a large number of unrelated tags, easily many thousands
  • Browsing these tags through a list, graph, or tag cloud, is not very efficient
  • Many efficient search strategies exist elsewhere:
  • All of these search strategies require knowledge of the relationships between tags
  • Logseq’s hierarchies of the form [[parent/child/teddy]] are not enough
    • Each child can only have one parent (teddy can be both in the child, and in the stuffedAnimals category, but currently this can’t be recorded)
    • The classification is specified on the pages themselves and can’t be added on later
      • If I tag 100 pages with teddy and later want to add the tag to a hierarchy, I need to edit every single page. Instead, it should be possible to tag pages, and then later classify tags centrally, making all of the original pages findable under the proper hierarchies

What needs to be done?

  • Logseq needs a way to specify relationships between tags:
    • TagA is a broader/narrower version of TagB
    • TagA is related to TagB
  • These relationships are captured centrally, such that tags can be managed without editing each tagged page individually
  • We need a user interface to hierarchically browse Logseq pages (not part of this feature request)

Example use case

  • Unsinkable Sam example goes here

Example implementation

  • This example uses Markdown to represent a subset of the Simple Knowledge Organization System, which is widely used, e.g. by libraries.

    • Tags are connected using the relations broader, narrower, broaderTransitive, narrowerTransitive and related
      • broader, narrower
        • specify that one tag represents a broader/narrower concept than another
      • broaderTransitive, narrowerTransitive
        • specify that one tag represents a broader/narrower concept than another and all its children/parents
        • e.g. A Cat is a narrower concept of a mammal, which automatically makes it a narrower concept of Animal
      • related
        • two tags are related, e.g. Apples and ApplePie
  • I am not tied to any specific syntax, this is just an example how it could be done in Markdown itself.

    • Alternatively, Logseq could directly parse SKOS RDF/Turtle description files.
    • SKOS is a minimal example, other knowledge management systems exist and in principle Logseq could record arbitrary relations between tags.
  • The following relationship is a (small) section of the animal taxonomy.
    Sub-items of the list are more narrower terms for their parent items. The lists can me arbitrarily nested. For example. Chordata and Mammalia are both narrower terms for Animalia. For a non-transitive relationship, Chordata would be a narrower description for Animalia, but Mammalia would not.

    • The animal taxonomy has the namespace animals to distinguish it from other hierarchies that can exist in parallel. One item can be in multiple hierarchies at the same time
    • 		  		  semanticRelation::narrowerTransitive
      		  		  concept::animals
      		  		  - Animalia
      		  		    - Chordata
      		  		      - Mammalia
      		  		        - Carnivora
      		  		          - Feliformia
      		  		            - Felidae
      		  		              - Felinae
      		  		                - Felis
      		  		                  - F. catus
      		  		                  - F. silvestris
      		  		  
      		  		  
      		  ```
      
    • This information is stored in a central Markdown file
    • If a user tags an item with animals:F. catus, the item will automatically appear in a search for Animalia
    • The user does not need to tag with the entire hierarchy animals:Animalia/Chordata/Mammalia/Carnivora/Feliformia/Felidae/Felinae/…, as this would duplicate the hierarchy on every item. The tag is only animals:F. catus, from which Logseq can infer that we are dealing with a type of cat.
  • Here is an example of a user-defined hierarchy that has non-unique names. It is not a taxonomy, but it is still useful to be able to capture such relationships, so probably this case shouldn’t be disallowed. If we require the user to provide minimal context, e.g. nutrition:Beef/Recipes instead of just nutrition:recipes, we can make tags unique. This will make any blocks/pages so tagged so visible under the Products, Meats, and Beef categories.

    ``` markdown
    	  semanticRelation::narrowerTransitive
    	  concept::nutrition
    	  - Products
    	    - Meats
    	      - Beef
    	        - Recipes
    	        - Nutrition
    	      - Pork
    	        - Recipes
    	        - Nutrition
    ```
    
  • This is an example of a “related” relationship. All of the tags [frying, deepFrying, airFrying, grilling] are marked as related.

    • If a user tags an item with the tag frying, a search for related items will bring up the other 3
      	  semanticRelation::related
      	  concept::cooking
      	  frying
      	  deepFrying
      	  airFrying
      	  grilling
      	  
      
      • Related tags can also live in different namespaces
      	  semanticRelation::related
      	  cooking:frying
      	  nutrition:fat