Advanced Query that pulls all reference AND recursive name spaces

[CappieXL]

I spend almost 12 hours on getting this working. So I’m sharing it with the internet hoping it saves someone else the same headache.

#+BEGIN_QUERY
{:title [:b "PAGINA Taken"]
:query [:find (pull ?b [*])
     :in $ ?currentpage %
     :where
     ; Unified condition: Either it's from namespace or it references the page
     [?b :block/marker ?marker]
     [(contains? #{"NOW" "LATER" "TODO" "DOING"} ?marker)]
     (or
       (and
         [task ?b #{"TODO" "LATER" "NOW" "DOING"}]
         [?b :block/page ?p]
         [?ns :block/name ?currentpage]
         (check-ns ?ns ?p)
       )
       (and
         [?p :block/name ?currentpage]
         [?ns :block/name ?currentpage] 
         (or [?b :block/refs ?p]
             [?b :block/path-refs ?p])
       )
     )
    ]
:result-transform (fn [result]
                  (let [sorted-result (sort-by (fn [h] (get h :block/marker)) > result)]
                    (map (fn [m] (assoc m :block/collapsed? true)) sorted-result)))
:group-by-page? false
:breadcrumb-show? false
:inputs [:current-page]
:collapsed? false
:rules [
[(check-ns ?ns ?page)
 [?page :block/namespace ?ns]
]
[(check-ns ?ns ?page)
 [?page :block/namespace ?t]
 (check-ns ?ns ?t)
]
]
}
#+END_QUERY

What does this query do?
This query pulls all the undone tasks that reference the current page somewhere in their path OR reside on any-level namespace below it. It shows them all.

How is it meant to be used?
Imagine you have the following hierachy: Work/Company A/Project X. You could paste this query on all three of the pages (Work, Work/Company A, Work/Company A/Project X) and it would show you all tasks related to that page including children pages from that point. It does not matter whether you tag a page inline, you indent under it or you put it on the page itself, it collects all those things.

So besides putting it on the pages itself you could do

• LATER From everywhere in Logseq, you can tag and it shows up in [[Work]]
• [[Work/Company A]]
  • LATER Task also show up both on pages ‘Company A’ and ‘Work’ when tagged like this
• [[Work/Company A/Project 1]]
  • We have a meeting here
    • Another bullet point
      • LATER This task is also found under ‘Work’, ‘Company A’ & ‘Project 1’ pages. As the depth does not matter

limitation:
Logseq doesn’t really handles aliases well. You’ll need to reference the entire namespace in order for the tasks to show up. So if you were to only tag [[Company A]], then it doesn’t show up (also not at page ‘Company A’, as it is really page ‘Work/Company A’. Logseq doesn’t internally link aliases like that. I really think this is a bug rather than a feature. Hope it changes sometime soon. Until then, we’re stuck with referencing the entire namespace always.

Edit: I just saw in the ‘Queries for Task Management’ thread that there is something like an (or-join) which could potentially solve this problem. I’ll leave that note here for now, I’m not gonna look into that myself for now.

Hope it helps anyone, cost me way to much time as ChatGPT doesn’t understand Datalog nearly as well as it does Python :')

[Siferiax]

NICE!
Linking it

Also I’ll grab this query and I’ll fiddle around and make the alias work (hopefully, that’s the plan at least)

One thing I do notice right away.

(or [?b :block/refs ?p]
             [?b :block/path-refs ?p])

That or is redundant. :block/refs result is contained in :block/path-refs. So :block/refs is not needed.

Also you define the marker values twice, which is redundant as well.
[task ?b #{"TODO" "LATER" "NOW" "DOING"}] isn’t necessary as you have [(contains? #{"NOW" "LATER" "TODO" "DOING"} ?marker)] already.

[Siferiax]

So ehm… I don’t wanna be mean or anything… butteh… what is the problem with aliases exactly?
Also your complex query can be made waaayyyyy easier… all the complex logic you did was redundant. (I expected as much, but wanted to test it first)

Here are some screenshots to better illustrate. So please test this out as well and let me know any problems you encounter so we can fix them!
At the top is the result from the query you posted and the query below is my simpler version.

And here’s the page I used initially / contains the actual tasks.

Here’s my version of the query:

#+BEGIN_QUERY
{:title [:b "Simple Tasks list"]
 :inputs [:current-page]
 :query [:find (pull ?b [*])
  :in $ ?currentpage
  :where
   [?p :block/name ?currentpage]
   [?b :block/path-refs ?p] ; this gets all the references to ?p regardless of the nesting.
   [?b :block/marker ?marker]
   [(contains? #{"NOW" "LATER" "TODO" "DOING"} ?marker)]
 ]
 :result-transform (fn [result]
  (let [sorted-result (sort-by (fn [h] (get h :block/marker)) > result)]
  (map (fn [m] (assoc m :block/collapsed? true)) sorted-result))
 )
 :group-by-page? false
 :breadcrumb-show? false
 :collapsed? false
}
#+END_QUERY

So love to hear the problem you have with aliases!

[Ngungu]

It’ll be interesting to see if and what he answers.

[CappieXL]

You’re not being mean at all. I highly appreciate the fact that anyone is even remotely engaged with something I invested so much time in.

Let me go over the differences.
Let us have the following setup:

It’s a hierarchy of Company/Project/Meeting. I’ve added one task tagged inline for every page, indented for every page and living on the pages itself. So 9 tasks in total (+2 alias-tagged tasks, I’ll come back to those at the end).

For the page: Company/Project/Meeting, our queries are the same:

For the page: Company/Project, my query has one task more. Namely the one that lives in it’s subpage “Meeting” itself.

Lastly, for the Company page, my query has two more: the task living on Company/Project and the task living on Company/Project/Meeting

All the additional namespace complexity was necessary to make the namespace search recursive. If i’m not mistaken, it’ll grab all the tasks of any depth hierarchical children.
I think I added the (or) around path and refs because I was under the impression that it would otherwise miss inline tags, but that doesn’t seem to be the case.
Lastly, as you can see, both our searches are missing the tasks that are indented beneath an alias-tag, which is a pitty. It tends to clutter my experience, but it’s not a biggie.

I did however, clean up my query based on your first two points of feedback. This is cleaner indeed.

#+BEGIN_QUERY
{:title [:b "Complex Tasks list"]
:query [:find (pull ?b [*])
     :in $ ?currentpage %
     :where
     ; Unified condition: Either it's from namespace or it references the page
     [?b :block/marker ?marker]
     [(contains? #{"NOW" "LATER" "TODO" "DOING"} ?marker)]
     (or
       (and
         [?b :block/page ?p]
         [?ns :block/name ?currentpage]
         (check-ns ?ns ?p)
       )
       (and
         [?p :block/name ?currentpage]
         [?ns :block/name ?currentpage] 
         [?b :block/path-refs ?p]
       )
     )
    ]
:result-transform (fn [result]
                  (let [sorted-result (sort-by (fn [h] (get h :block/marker)) > result)]
                    (map (fn [m] (assoc m :block/collapsed? true)) sorted-result)))
:group-by-page? false
:breadcrumb-show? false
:inputs [:current-page]
:collapsed? false
:rules [
[(check-ns ?ns ?page)
 [?page :block/namespace ?ns]
]
[(check-ns ?ns ?page)
 [?page :block/namespace ?t]
 (check-ns ?ns ?t)
]
]
}
#+END_QUERY

[cloudy610pencil]

@CappieXL

check

@CappieXL
wow, this is great Thanks a lot for sharing this i was looking exactly for this, is it possible to use this query instead of the current page, can I give namespace name and get all tasks under the namespace. if it’s possible, I am planning to use it on my journal page where i can have project-wise tasks can listed on journal page

For example

Work/project/1
Work/project/1
Work/project/1


Personal/home
Personal/travel
Personal/save

if i want tasks under work can be listed ?

[cloudy610pencil]

am a rookie on queries but just found it by trail & error
seems just I have to change the :inputs ["work"] on the query now I can query all tasks from work namespace

[Siferiax]

@CappieXL

Namely the one that lives in it’s subpage “Meeting” itself.

Ah that makes sense. I guess I missed that part somewhere!

@CappieXL

All the additional namespace complexity was necessary to make the namespace search recursive. If i’m not mistaken, it’ll grab all the tasks of any depth hierarchical children.

Yes, references go up (toward parent), but not down. I’m interested in taken another look some day again

@CappieXL

Lastly, as you can see, both our searches are missing the tasks that are indented beneath an alias-tag, which is a pitty.

This is definitely solvable with a or page or alias of page. I’ll take that on when I come back to this. For now here’s an example of such a thing.

Queries for task management

I’m going to gather some queries here that will be useful for task management. It won’t be done in one go and I will be adding inline comments to help you understand what is going on. I have added those to build on the queries in order. So read through them all as I have only added comments the first time a statement appeared. Hopefully this will help you be able to edit the queries for yourself. Tasks past due Tasks scheduled value is past due (based on today) #+BEGIN_QUERY {:title [:h3 "…

And one in the context of nested todo’s.

FInd nested TODOs

Since it was featured in Logseq Times, I’ll just add to this post to say I’m a dumdum and there is an attribute for this, making the query A LOT simpler! This is for the query based on page reference, not the one based on parent blocks. #+BEGIN_QUERY {:title ["Query by page & alias"] :query [:find (pull ?b [*]) :in $ ?page :where [?b :block/marker "TODO"] [?p :block/name ?page] (or-join [?b ?p] [?b :block/path-refs ?p] (and [?p :block/alias ?a] …

[ddavo]

You could avoid the [?ns :block/name ?currentpage] clause in the second and by using an or-join instead (learned it from @Siferiax)

#+BEGIN_QUERY
{:title [:h3 "Tasks within recursive namespace or reference" ]
 :query [:find (pull ?b [*])
    :in $ %
    :where
      [?b :block/marker ?m]
      [(contains? #{"TODO" "DOING"} ?m)]
      (or-join [?b]
          (and
             [?b :block/page ?p]
             [?ns :block/name "anki"]
             (check-ns ?ns ?p)
          )
          (and
             [?p :block/name "anki"]
             [?b :block/path-refs ?p]
          )
      )
 ]
 :collapsed? false
 :rules [
      [(check-ns ?ns ?p) 
       [?p :block/namespace ?ns]
      ]
      [(check-ns ?ns ?p)
       [?p :block/namespace ?t]
       (check-ns ?ns ?t)
      ]
  ]
}
#+END_QUERY

Oh, and I almost forgot, if you end up using ?currentpage, remember to add it to the or-join list!

[paulrudy]

EDIT: I thought I came up with a solution for the aliases issue, but it doesn’t quite work. Removed the code I posted, but the essence was this: From @CappieXL’s updated code I added a third item to the (or ...) to capture aliases:

(and
        [?p :block/alias ?a]
        [?b :block/path-refs ?a]
        (check-ns ?ns ?p)
      )

But the problem is that the results of this are global. Couldn’t figure out a way to limit them to aliases of the current page’s namespace.

[Siferiax]

• Changed some things to add the alias.
• Changed the variable name to inputpage as that is the best description of what it is, regardless of whether you use :current-page, :query-page or a hardcoded name.

#+BEGIN_QUERY
; see https://discuss.logseq.com/t/advanced-query-that-pulls-all-reference-and-recursive-name-spaces/21275/8
{:title [:h3 "Recursive Namespace + Alias Tasks" ]
  :query [:find (pull ?b [*])
    :in $ ?inputpage %
    :where
    [?b :block/marker ?m]
    [(contains? #{"NOW" "LATER" "TODO" "DOING"} ?m)]
    (or-join [?b ?inputpage]
      (and ; Is de page the task is on part of the namespace of the input page?
        [?b :block/page ?p]
        [?ns :block/name ?inputpage]
        (check-ns ?ns ?p)
      )
      (and ; Does the task refer to the input page in its lineage?
        [?p :block/name ?inputpage]
        [?b :block/path-refs ?p]
      )
      (and ; Does the task refer to an alias in it's lineage, either of the input page or a page in its namespace?
        (or-join [?p ?inputpage]
          (and
            [?ns :block/name ?inputpage]
            (check-ns ?ns ?p)
          )
          [?p :block/name ?inputpage]
        )
        [?p :block/alias ?a]
        [?b :block/path-refs ?a]
      )
    )
  ]
  :result-transform (fn [result]
    (let [sorted-result (sort-by (fn [h] (get h :block/marker)) > result)]
    (map (fn [m] (assoc m :block/collapsed? true)) sorted-result)))
  :group-by-page? false
  :breadcrumb-show? false
  :inputs [:current-page]
  :collapsed? false
  :rules [
    [(check-ns ?ns ?p)
      [?p :block/namespace ?ns]
    ]
    [(check-ns ?ns ?p)
      [?p :block/namespace ?t]
      (check-ns ?ns ?t)
    ]
  ]
}
#+END_QUERY

@CappieXL Sorry for never getting back to it D: (or at least not sooner)

[paulrudy]

Thanks for this! Works perfectly. I think I understand how the second or-join you added is working. I’m happy that I was getting close with my attempt . Makes sense to change the variable name.

I saw that @CappieXL had changed their strategy. I wonder with this working, how people see the advantages and disadvantages to the two strategies?

Using query-page with this one means the query could be embedded anywhere and capture all tasks for the namespace, which is nice.

[mentaloid]

@paulrudy

I wonder with this working, how people see the advantages and disadvantages to the two strategies?

From what I understand, your question is not about Datalog, but about using namespaces (e.g. Company/Project/Meeting) or not. Namespaces:

• are inferior in all of:
  • writing references
    • They need more typing.
  • reading references
    • They take more space.
  • querying references
    • They need recursive rules (like in the present thread).
• add complexity that is both:
  • unnecessary
    • By the amount of hours spent for the present thread.
    • By Occam’s razor.
  • limiting
    • Sooner or later, their single-parent nature causes maintenance problems.
      • e.g. a meeting may be about more than one project or company etc.
• are discussed a lot in this forum
  • Could follow the links in this thread.

[paulrudy]

Thanks for that. You’re right, my question does boil down to namespace vs other strategies of connecting and grouping topics and notes.

@mentaloid

Sooner or later, their single-parent nature causes maintenance problems.

This limitation in particular is already clear to me. Then starts a clutter of aliases and tags try to work around the limitation.

[archer_loaf]

thanks so much for this! How would I sort the resulting list by schedule and/or deadline date?

[prince]

is it possible to modify it for journal pages instead of normal pages?

Let’s say i want to pull all blocks (instead of only tasks) that are tagged/referenced with journal pages of last week (any day from last 7 days). These blocks are shown as linked references on their respective journal pages.

[paulrudy]

try replacing the :result-transform line with

:result-transform (fn [result] (sort-by (fn [h] (get h :block/scheduled)) > result))

I’m not sure how to do both scheduled and deadline, and I’m also not sure how to affect the sort order (descending vs ascending)

[paulrudy]

This seems complex. It might be worthy of a separate topic.

[prince]

luckily i realised later that i may not really need this kinda query

[mentaloid]

@paulrudy

I’m not sure how to do both scheduled and deadline

Could nest one as not-found fallback of the other, e.g. (get h :block/scheduled (get h :block/deadline))

@paulrudy

I’m also not sure how to affect the sort order (descending vs ascending)

This one is tricky. But if you achieve the inverse order, can simply wrap it inside reverse.