"Introducing 'Views': A Flexible Solution for Graph Organization and Content Separation in Logseq"

I propose a feature I will refer to as “Views”.
This will solve specific problems that have been discussed in topics on this forum but in a flexible way. See Separate graphs for work and home? - Questions & Help - Logseq for background.

  1. Typical user starts off with the typical graph structure and everything at root level as normal:
- Logseq	  
  - assets
  - journals
  - logseq
  - pages
  - whiteboards

A new folder is introduced (this is to demonstrate the concept not the final UI)

 - views

The user can create a view of their graph by defining a filter query and giving it a name. It then shows up under “views”…

 - views
  - My work stuff (this is expandable)

When you click on this query you don’t just see a page with the results of the query, rather it acts more like a node a parent node in the graph and projects all the matching items of the query as nodes in the subgraph i.e like a top level folder. So you can expand it and so your total structure now looks like:

- Logseq	  
  - assets
  - journals
  - logseq
  - pages
  - whiteboards
  - views
    - My work stuff
       - assets
       - journals
       - logseq
       - pages
       - whiteboards
     

However in this expanded view, the only nodes present are the nodes that satisfy the query for that “view”…

This could be made even more powerful in future, with things like:-

  • The ability to publish, export, views.

This would solve the work / personal separation because, or even just situations in which you want to prepare for what can be seen at some presentation, talk or demo:-

  • Those that want physical seperation can still use seperate graphs and ignore this feature.
  • Alternatively those that want top level folder seperation can have seperate “parent” pages in the hierarcht for work vs personal, and can then define a seperate “View” for “Work” that just queries everything from the “Work” folder. This is less fragile than relying on metadata and only needs to be set up once. When in a work presentation, the user is able to switch to the “Work” view, and this constrains the graph that logseq is displaying in the app and that the current user can work with, until they switch out of this view, to the default view or another. So liss danger of work content being visible during that presentation.
  • And for those that are comfortable purely with metadata and queries to distingush work / personal content, can forgo structuring there pages into top level /personal / work folders and can just rely on the metadata in their queries, and still leverage views to switch contexts if desired.

Just because we already have queries, it doesn’t mean that they are the right tool for creating the desired secure views. Filtering, and more generally starting with the whole and then separating it down to parts, presents all kinds of challenges and risks (never-ending global security mess testifies to that). It is not easy to:

  • make a composite query work
  • guarantee that a working query works exactly as desired
  • make it performant for big graphs
  • make a small change without breaking the above
  • explain that to and educate non-technical users
  • make a major update to the application without breaking the above

Should instead go for composition, which is the inverse:

  • Start with graphs as separate as possible (even physically separated).
  • Develop a technology that treats multiple graphs as if they were one graph.
    • This is the one-time hard part.
  • Then to create a view becomes as simple as to:
    • pick the desired graphs
    • optionally arrange them to determine the behavior in case of conflicts

When that is ready, there is minimal distance to also combine graphs from different users.

3 Likes

I’m not sure synchronizing and merging different graphs into one would necessarily be any simpler than queries. Besides, such an approach would require to preemptively structure the different graphs around the views that you want to have, which is something that is likely to change over time and that a user might not know in advance. I don’t think it’s always easy to know which graph a piece of information belongs to, making ideas like quick capture harder to implement in this scenario.

There can be some cases where the queries used might be overly complicated and underperforming, but I feel like it shouldn’t really be that way for easy ones, such as filtering based on a tag or property, which I guess would be the common case.

I think having the common case be well performing and the hard case be slow but possible wouldn’t be bad. But in case that’s too hard, maybe some simpler filtering mechanisms could be put in place, ex. just specifying tags or properties. Although queries “feel” like they should be the right tool for this.

@mentaloid I think both approaches are somewhat complimentary.
However with the composite approach, the user has to seperate there content into seperate graphs from the outset, with the eventual use case in mind which cannot always be anticipated in advance. It’s also the case that when they want that contextual view to give some presentation, or talk at some hobby group, it may be that the boundaries of the view, and the boundaries of whats in each graph don’t perfectly align so they have to physically create graphs to model their desired View… sounds nasty. i.e I keep alll my hobby information in a seperate graph and luckily for me I end up talking at a hobby group one day, so I can view just that graph. But then I take up another hobby and some of the content in the graph is relevent for one hobby session, but not another. Do I have to split it our again? What about when there is overlap - i.e at some point I want the freedom to flexibly model what I can see / access, not in terms of an entire graph r composite of graphs, but rather querying the graph / or accross the graphs, so whether thats a single graph (as this feature proposes) or mulltiple (you’ve basically tacked on a complimentary feature but one that doesn’t necessarily preclude the solution for the problem I am suggesting).
I have removed the mention of “sharing” from the proposition, because I think that is muddying the waters.

Short version

Are we trying to solve the same problem? Remember that this discussion begun with the problem of a high-level work-home or public-private separation. I have a few questions:

  • Would you ever attempt that separation at block-level?
    • Wouldn’t that be:
      • impractical?
      • risky?
      • difficult to implement?
  • Don’t all graphs already group some pages together?
  • What do you “feel” closer to a secure virtual view of pages?
    • a specialized virtual container of pages included explicitly with no surprises ever
    • a generic filter of blocks included after calculations that:
      • can return anything between:
        • the empty set
        • the set of every page
      • a typo can easily change (even invert) the result
        • e.g. > instead of <

Long version

I should have guessed that my suggestion would be read from the perspective of what we have today. Let me clarify:

  • It is not about fixing performance.
    • Problematic performance is merely an indicator of a problematic solution.
  • It is not about bringing back a folder-structure.
    • That would be a big leap backwards.
  • It is not about going down to one page per graph.
    • That would defeat the concept of graph and its benefits.
    • Graphs are much more than collections of pages.
      • They are like libraries of cohesive books (one per Logseq page), within a specific domain of knowledge.
      • But these books are interconnected.
        • This can provide enormous leverage.
  • It is not about planning ahead some structure.
    • That would be very rigid and inflexible.
    • Structuring graphs should always be around their core nature (i.e. by affinity), not around a specific usage (should support endless usages).
      • Users don’t need to know any particular usage in advance.
      • Views are the ones structured on-demand around a particular usage, combining graphs according to that usage’s needs.
        • Views may change all the time, they are not permanent at all.
          • Though they could support meta-pages, if needed.
        • The challenge is how to better generate those views:
          • With queries.
          • With graphs.
  • It is not about syncing graphs.
    • Different graphs don’t need any syncing, only same graphs need.
    • We have enough troubles with syncing as it is.
  • It is not about sharing pages among graphs.
    • Each page should belong to a specific graph.
    • Repetitions are heavily discouraged.
      • When writing a page, there is always a single target-graph.
        • A page may always be put at a wrong graph and queries cannot prevent mistakes either. The goal is to make the most costly mistakes (those of security) less easy to commit.
  • It is not about merging graphs at the data level.
    • That would add excessive overhead on every change (among other issues).
    • Merging the desired graphs should only happen virtually, at the usage level.
      • When working with combined graphs, all the functionality of a single graph is still there.
      • That includes:
        • navigation
          • This is mostly about preventing navigation outside the desired combination.
        • queries
          • They run on every graph in the combination, then the merging applies to the results.
        • exporting, publishing etc.
          • A graph can be served both by files and by databases. Likewise, a virtual graph can be served both by one and by multiple underlying graphs.

Being inverse to each-other, the two suggestions may seem complimentary, but for me they are not.

  • They target the same problem with the same concept of views, so one is redundant.
    • When all you have is queries, everything looks like a problem solvable by a query.
    • But simple problems should be solvable by simple tools.
      • If a simple problem is not solvable by a simple query, then queries are an inferior solution.
      • Separating things should be as simple as putting them in separate containers.
      • Mixing everything into a big container and using queries to form virtual containers, may work, but is the wrong tool.
      • Working with whole containers is both simpler and faster.
      • Putting smaller containers inside bigger containers (in endless combinations and without binding any of them) is intuitive:
        • In different trips we take different things with us. We could either:
          • filter the whole house to decide what to take with us
            • This is necessary if our house is a mess.
            • We still have the feeling that we missed something.
          • just take directly what we actually need
            • This works if our house is in order.
            • We can easily verify the process.
  • If dealing with overlapping graphs can be difficult (which is debatable), what makes you think that dealing with everything at once is easier?
    • The same reason that prevented the clean separation of graphs, will prevent the creation of a straightforward query.
    • If both are imperfect, the questions are:
      • Which one is more predictable?
      • The extra pages of which one can you afford to ignore?
  • Graphs get modified (new notes get added, new connections are made etc.)
    • If maintaining pages in appropriate graphs can be hard (which is debatable), maintaining composite queries can be a nightmare.
      • The worst scenario may be when a trusty query fails to cover a new case:
        • It may be too late when you find out.
        • Even after you find out, fixing the query remains tricky.
          • You hardly know if your new fix will cover a future case.
        • Every time you add a sensitive page, what is easier?
          • Simply avoid putting it in a “dangerous” graph.
          • Think about every “dangerous” view whether it filters it or not.
            • Keep adding/removing tags until you are confident enough.
        • You cannot forget putting a page somewhere. But you can forget tagging it against some view.
  • If you think that querying is easier than splitting, then have the query do the splitting for you.
    • But if you cannot find a query that does the exact splitting, you will either:
      • give up
      • do the work manually
        • When that happens, you would wish that you could work with an already smaller graph, instead of facing every single page that you have.
  • What if following both approaches?
    • That could result to the worst of both.
  • The fact that one suggestion contributes to sharing graphs among users, doesn’t change its primary target (that of effective content separation), it is merely a clue that it has the right direction.

Queries are overpowered and yet overrated.

  • Queries cannot be more useful than the data they run on:
    • Some people think that they can throw notes all over the place and let queries (or AI) bring them back. That is simply impossible:
      • It is not what queries are made for to begin with.
      • In the end, what takes place is some sophisticated look-up, which covers most cases.
        • But can you afford the remaining ones?
        • Even query experts (which are rare) cannot cover every case, unless they have good knowledge of the underlying graph.
    • For queries to be either simple or performant, the hard work has already taken place during tagging etc.
    • Just because we have easy tagging, it doesn’t mean that it is the right tool either:
      • Separating views by tagging is an enormous work.
        • And yet tagging with the name of a graph could automatically move this page into that graph.
          • There are more ways, but:
            • this post is already lengthy
            • the point is that the solution doesn’t have to come from within the problem’s limited frame
      • When the tagging is imperfect, queries can become:
        • more complicated
          • Things can get very ugly, especially in bigger graphs.
        • less useful
          • As graphs get bigger, notes get practically lost and you don’t even know that they are missing from the results.
            • For many users, losing notes is not a problem, it is a feature, because they use Logseq mostly to dump their brains. These people anyway:
              • don’t need content separation
              • could simply dedicate a specific graph to dumping
            • But for many users, every single note counts, especially the sensitive ones.
              • These people need better organization and thus better tools.
                • Better doesn’t mean more powerful (thus more dangerous), it means more fitting.
3 Likes

What I’m thinking is, check what a block is tagged with, and if the tag matches one of those that should be hidden hide it. You seem to be thinking that this would be harder/more impractical or insecure than putting together a bunch of graphs in memory, and I’m not sure I really understand why. I think we might not be understanding each other here. I feel like loading private data together with public always puts you at risk of showing the wrong thing at the wrong time, depending on the implementation.

I can agree that the full power of simple queries might be too much on this front. Maybe it would be a good idea to limit this to tagging/properties excluding more general queries.
I can also agree on the fact that it could be debatable on wheter this should be done at the block or at the page level, also depending on what is more practical/achievable from a development perspective.

I do although think that it would be way more impractical, on the user perspective, to have a bunch of graphs, each one with its own journal, and to have to switch graph for every topic switch the user goes through (which is kind of what I’ve been doing in the past). I understand you want to be able to work with multiple graphs at the UX level, but then I feel like you’d have the same problems you’re thinking of when you talk about a metadata-based solution. In short, when you create a page/journal block in the multigraph, you’d have to pick a graph it belongs to, and you’d need to keep mental track of what graph contains what to avoid showing something you wouldn’t want to be seen. The difference is, if we’re talking about different containers, things can no longer be intersectional: for example, something cannot be both in the “private” graph and in the “psychology” graph, I’d have to pick one and I feel for some things there just wouldn’t be right choices, just wrong compromises.

Would it? I feel like it would be zero work for me. I already tag my pages with the topics they belong to. To have something that is more generally private, I could just type #private at the end.

Besides, a metadata-based approach would work on existing graphs which are often already tagged, while what you’re describing would require splitting a lot knowledge bases, something I already had to do in the past to have published graphs corresponding only to certain topics. I feel like it is kind of absurd that I have to do this by hand or by grepping files, when the metadata about what page belongs to what topic is already there for me.

And, of course, this wouldn’t be the only time such a restructuring would be required. Because it is, you know, a restructuring, so you can say this:

But of course putting certain things in separated graphs would impose a very rigid structure on the information you put in, which is the choice of singular graph it should belong to (even though most things almost always cross topic boundaries, as you’d want to be represented in a tool for interconnected thought).

And yet, what you’re proposing would bind views and structure, inextricably tying the information I put in (ex. my psychological state) with its usage (whether I want to keep it private or not). On the other hand, metadata allows me to just tag something with both #psychology and #private, as well as many other approaches that would fit a specific individual better.

I think it’s good that we agree on this. But then this seems to be contradictory to me:

I think I’m missing something here? This to me feels exactly like folders, an hierarchy of containers. Please let me know if I’m getting this wrong.
Besides, my house will always be a mess. I want a tool that embraces that.

I have the same impression, so I’m not going to answer everything.

Certainly. Maybe you read too much into the house-trip example or the word container. To clarify, I’m going to rename containers into graph-sets. Graphs have many restrictions that graph-sets don’t. Compare these two:

  • In a folder-like hierarchy:
    • every node (except of the root) belongs to one and only one other node
    • multi-level nesting produces branching
      • This is the source of problems, as a single structure tries to cover all possible needs:
        • Folders need planning.
        • Usage needs discipline.
        • Deleting, moving or even renaming a folder can easily break things.
  • In contrast, with graph-sets (ex-containers):
    • graphs (and graph-sets) don’t belong to any graph-set, they move around as needed
    • unlike in the physical example:
      • all combinations of nested graph-sets always produce a single flat graph-set
      • a graph or graph-set can participate in multiple graph-sets at the same time
        • even if those graph-sets participate in the same bigger graph-set
          • The contents of each graph-set don’t affect any other.
          • Graph-sets don’t have to be reused.
            • They can be modified, deleted etc. at any time without affecting anything.
            • Users don’t need to plan graph-sets for the future, they just put together something that covers their present need.
            • Each need could even have its very own graph-set.
              • Even using two identical graph-sets under a different name would be fine.

Many tools can operate in messy conditions (the real “wrong compromises”), but never in a secure way.

  • With queries, it is needed to:
    • always load everything
      • This is inherently vulnerable to leaking sensitive info.
    • then filter the sensitive parts out
      • multi-tagging is a hidden form of tying
        • Every additional tag is like one more knot that filtering needs to consider.
        • At the same time, missing tags affect a lot the quality of tag-based queries.
        • Generally, heavily-tagged graphs are discouraged, because:
          • they are difficult to maintain
          • their noise hides much of their value
  • With separate graphs, the principle is to load:
    • when in public: only non-sensitive graphs
      • When something is not loaded, it is impossible to leak.
    • while in private: mixed (or even all) graphs
      • Everything loaded is available like if it was a single graph:
        • There are no boundaries.
        • No information is tied.
        • This is a prerequisite, otherwise the solution is wrong.
      • One graph has to be the target for new pages. As I showed:
        • both tagging and picking a target-graph need the same effort
        • yet tagging is more error-prone, because it allows everything
          • It is exactly the overflexibility of tagging and the overpower of queries that are:
            • good for finding info
            • bad for protecting info
1 Like

Thank you for the thorough answer, this clarified a lot. In the end, I think this comes up to a difference in philosophies and I see where you’re coming from.

@mentaloid
In terms of the strict boundary / isolation for security purposes - this is exactly why I use two seperate graphs today for personal versus work content and is what I advocated for on the thread I linked.

However for the use case I had in mind with this feature proposal, it was really all about not enforcing that requirement / answer onto the user who isnt after the strict security but more the convenience of a contextualised view perhaps for some adhoc presentation. You’ve asked some questions many of which seem to come down to the “safety” in terms of security around queries versus graphs. For my particular work / personal scenario, my preference is that I want the security guarantees of not accidentally mixing content… but for others it might not be necessary. Keep in mind Work versus Personal is only one case. To someone who owns their own business they might not initially see any case for splitting this into two graphs. There are other cases like for example:-

  • Someone who gives Demo to customers who might be asked to share their screen and they want to show that customers contents without for example - work meetings being present.
  • Someone giving presentation on a project and want only that projects notes to be visible for the duration.
  • Someone giving presentations at hobby groups. They may be part of several different hobby groups, and on Thursdays they just want their “Beer brewing” content to be shown, and Friday’s its “Whisky club” content time. Both areas have some overlapping content.

These people may be comfortable with accepting the tradeoff in terms of risk in security, versus having to prepare and maintain seperate graphs. You would hope they would enact simplfiied query or stucture mechanism to reduce that risk as much as possible. For example perhaps they resort to namespace per customer or something. To me the feature I proposed was the one about a user wanting to eliminate the noise from the UI for a contextualised view of the world, not a hard security feature - as the latter is already there - I can and do already use seperate graphs.

Saying all this, I could imagine support for combining or attaching graphs indirectly enabling this feature but not by eliminating the query aspect:

  1. User defined a query to filter context for their “contextual view”
  2. The matching nodes is built into its own graph
  3. The graph is “attached” as a subgraph and shows up under a “Views” folder.
  4. Imagine the view now being selectable in the dropdown where you can select what graph you have open.

But… this is pretty much the feature I described, a seperate graph is part of the solution, but its built by an adhoc query that can meet the user where they are and not punish a user who hasnt split their content into graphs from the outset to match a potentially future adhoc requirement, and not require the original content to be maintained any differenly moving forward than it was before. In other words, after the presentation is done, the view could be deleted with no impact.

1 Like

If the title is changed from promising things that queries cannot do, namely:

  • content separation
  • graph organization

to instead promise:

  • content hiding
    • inherently insecure
  • through graph filtering
    • the natural application for queries

then I can agree that this is:

  • a separate feature
  • with useful applications in some scenarios

@mentaloid
Agreed, good suggestion. I outsourced the title creation to ChatGPT and having had this discussion I am inclined to agree it’s not the best. Thank you for all of your detailed feedback thus far.
I can’t seem to figure out how to change the title!
I was thinking of something like “Contextual Views for Presentations” or maybe “Switching to a filtered version of the graph for a presentation” - if anyone is able to adjust the title please do.

1 Like

A small thought. When it comes to security its sometimes useful not just to think in terms of boundaries (i.e seperate databases, seperate web apps etc), but also there are some boundaries that have their own security model within them. For example, a SAAS web application, will let a user authenticate, and will use a query to work out what roles and permissions that user has. They then may have access to only the parts of the web application which is relevent for their role / permissions (Authorisation). The UI is presenting what’s available to them, based on a query that takes their roles and permissions into account.

  1. If there was a mistake in this query, a user may well see something that they are not meant to.
  2. If an admin had assigned the wrong role or permission to the user, same result.

Now, LogSeq doesn’t have its own internal security model, it’s a single user application and the user of the app has full “rights” to all pieces of content, which includes CRUD operations.

If LogSeq did have a security model, there are many ways it could be implemented.

  1. Seperate User Profiles would take on the role of the different “Contextual Views for presentation”
  2. Content blocks or pages could have strict metadata relating to the security model feature - i.e The profiles or permissions that the “user” needs in order to peform CRUD operations.

In LogSeq if this feature was enabled it could:-

  • have strict UI’s to ensure such content had valid security model attributes reducing a certain class of error such as “typos” in attribute names.
  • Assume a “restrictive” rather than “permissive” default policy - i.e any content without appropriate security model metadata would only be accessible under the “root” profile.

This was a thought experiment to think through how such a security model might be implemented and could achieve the same goal, without necessarily introducing seperate graphs… but ultimately at some level its still query based as it’s now able to achieve not just “content hiding” but also preventing accidetnal CRUD operations during that presentation etc, based on the security model, and the queries powering that feature.

Mind the KISS principle.

I value simplicity and this would add complexity to Logseq and make maintenance harder.

This is my solution to the problem to be able to separate work and home. Create 2 separate graphs. I have a “home” graph and a “work” graph. Sometimes they intersect and then I just add a link to the other graph. I have 2 windows open of Logseq which I can easily switch between. I have also added some custom.css to make a distinct color of the left panel to be able at an instance to know which graph I am looking at.

1 Like

@gh_hy44
The proposed feature wouldn’t add any complexity for you - simply don’t use it if you don’t need it. Your usage would remain just as simple as before.

  • Queries, already exist.
  • The ability to naviigate a graph of content in the UI already exists.

I don’t see any massive complexity added here, maybe you can elaborate?

I also use seperate contexts for work and home. I am not able to edit my initial opening post, but this wasnt just about that one specific case. If you read above, there are examples where the user wants a contextual view (for example for some presentation) where its impractical to maintain content in seperate graphs from the outset or on an ongoing basis. If you reject the use cases or the proposed solution still then fine, would be good to know:-

  • Do you think the problem attempting to be solved is just invalid or already has an acceptable solution?
    • Maybe one should build / prepare a seperate graph manually per presentation? Or is there some way of using a query to do a filtered export of content perhaps in order to build a seperate graph? Think of the person with multiple hobbies who wants to give a presentation at one hobby group on Monday and another on Thursday and hasnt setup two seperate graphs from the outset

I can’t seem to edit the opening post and I wonder if it would be best to close this topic and open a fresh one which attempts to be more refined from the outset, to make this not about content seperation but content hiding as @mentaloid suggested.

1 Like

Notice that this can partially be implemented manually by using “nested graphs” but it works only on desktop OSs that support symlinks. The point is creating some graphs and then one that contains symlinks to the /pages folders from other graphs. It works if you don’t have pages with the same name in different graph and this can be solved using namespaces and you can also create a “base” graph containing only “meta” pages like the pages for properties that you may want to be in common between all your graphs (again by using symlinks).

It sounds complex but you can setup it in 5 minutes starting from scratch while adapting your existing graph to this may takes some time.

That’s a clever use of symlinks and worth knowing about, thank you for sharing!

I just wanted to add clarification that the concept of composing graphs versus “views” are complimentary but different features.

The graphs bit, helps with providing the raw material / data for a query.
The “views” bit, is about flexibility over what is presented from those graphs, whether that be:-

  • a single graph or a composite
  • whether a composite graph is achieved using symlinks or some other newly introduced feature

When creating a View:-

  • You are free to select any nodes of the graph (be that a composite or single graph) using query capability.
    • This means you may achieve things like: select page foo - (originates from graph A), and page bar (originates from graph B), but filter out everything else.
    • You could clone that view adhoc, and modify the query to also include some other page from graph C.
  • You can create as many views for your use cases as you like, perhaps including overlapping sets of pages / blocks, or perhaps not, and delete them when done.
  • The queries for the views can meet your content where it is. Either by selecting specific pages or blocks, or using tags etc
    • This means it will suffer from all the usability issues people have with queries.
      • LogSeq solution to improve that situation on the query authoring side slightly is it’s “simple query” experience. Perhaps that will improve over time.

If it was for me, Logseq should expose the results of all the queries as a virtual filesystem, maybe a WebDAV local server since it’s supported by all the three desktop OSs. Maybe let you set each query to be “mounted” as a specific folder. Those virtual files would be available as far as Logseq is running, but to make them available when it is not, it could create folders containing symlinks to pages that are query results.

To solve the problem of pointing to a specific block using a file, at least on Linux one can create a .desktop launcher that open the default text editor at a specified line.

1 Like