Pages with a property value AND a header matching substring

Questions & Help Queries
1 
#+BEGIN_QUERY
{:title [ :h2 "Missing Pages" ]
 :inputs ["concept" "Notes"]
 :query [
    :find (pull ?p [*])
    :in $ ?ptype ?header
    :where
        (page-property ?p :page-type ?ptype)

        [?b :block/page ?p]
        [?b :block/content ?bcontent]
        [(clojure.string/includes? ?bcontent ?header)]
  ]
}
#+END_QUERY

This is my query so far, it should find pages that both:

  • have page-type:: concept (this bit works)
  • have a header (any level) that matches # Notes

There is at least one page that matches, but it’s not showing, and I’m not really understanding how the last chunk of the where clause should be structured.

2
  • Your query works to me.
  • In an external editor, check the markdown of the page that doesn’t show.
  • Check also the console (Ctrl + Shift + i) for any error messages.
1 Like
3

You’re right, it is working. My graph was not right :confused:

But I’m still stuck, because this query was a stepping stone to a different query, for pages that:

  • have page-type:: concept (this bit works)
  • DO NOT have a header (any level) that matches # Notes
  • (and eventually, so extend that do not have to include multiple headers (so the search returns any pages that are missing any one of those headers.)

The next iteration the not is stumping me:

#+BEGIN_QUERY
{:title [ :h2 "Missing Pages" ]
 :inputs ["concept" "# Notes"]
 :query [
    :find (pull ?p [*])
    :in $ ?ptype ?header
    :where
        (page-property ?p :page-type ?ptype)

        [?b :block/page ?p]
        [?b :block/content ?bcontent]
        (not [(clojure.string/includes? ?bcontent ?header)])
  ]
}
#+END_QUERY

This returns all the page-type:: concept pages, including the one with the # Notes header. I guess because all pages contain at least one block that doesn’t match the header?

Is there a way to find pages where all blocks don’t contain a string?

I tried this:

        [?b :block/page ?p]
        [?p :page/blocks ?pageblocks]
        [?pageblocks :block/content ?bcontent]
        (not (some [(clojure.string/includes? ?bcontent ?header)]))

But I got error: Join variables should not be empty, and I don’t have much of an idea how to fix that.

4
  • Your attempt means this:
[?b :block/page ?p]                  ; and there is at least one block in this page
[?b :block/content ?bcontent]                        ; the content of which
(not [(clojure.string/includes? ?bcontent ?header)]) ; does not include that header
  • Should rather make the (not) wrap the whole paragraph:
  (not                                             ; and there is not
      [?b :block/page ?p]                          ; any block in this page
      [?b :block/content ?bcontent]                  ; the content of which
      [(clojure.string/includes? ?bcontent ?header)] ; includes that header
  )
1 Like
5

Thanks again @mentaloid, super helpful explanation.

The final query I settled on for multiple headers is:

  #+BEGIN_QUERY
  {:title [ :h2 "Pages missing multiple headings" ]
   :query [
      :find (pull ?p [*])
      :where
        (page-property ?p :page-type "concept")
  
        (not
          [?b :block/page ?p]
          [?b :block/content ?bcontent]
          (and
            [(clojure.string/includes? ?bcontent "# Notes")]
            [(clojure.string/includes? ?bcontent "# Related concepts")]
          )
        )        
    ]
  }
  #+END_QUERY

I tried doing a for loop inside the and, but couldn’t make it work. But this solution is good enough for now :slight_smile:

I’m using this as a kind of rudimentary documentation-format test, to ensure that all my concept pages conform to a similar structure. I’m sure there’d be nicer ways to do this, but this is very useful.

6
  • This would imply an or-clause.
  • By using an and-clause instead, you rather get “any pages that are missing blocks with both those headers”.
7

Urgh, you’re right. However, simply swapping in or doesn’t fix it:

  • (not ... (and ... gives any page that’s missing BOTH headers
  • (not ... (or ... gives any page that’s doesn’t have at least one of the headers.

Neither is what I want, which is any page that is missing either of the headers

I think then I need something like (or ... (not ..., , but I don’t know how to structure it. I’ve tried:

        (or
         (
          [?b :block/page ?p]
          [?b :block/content ?bcontent]
          (not [(clojure.string/includes? ?bcontent "# Notes")])
          )
         (
          [?b :block/page ?p]
          [?b :block/content ?bcontent]
          (not [(clojure.string/includes? ?bcontent "# Related concepts")])
          )
        )

(I get Join variables should not be empty)

and

        [?b :block/page ?p]
        [?b :block/content ?bcontent]
        (or
          (not [(clojure.string/includes? ?bcontent "# Notes")])
          (not [(clojure.string/includes? ?bcontent "# Related concepts")])
        )

all pages get returned.

Not sure if I’m hitting the limits of my clojure understanding or my basic capacity for logic :weary:

I think the bit I’m missing is that I need to do something like a clojure filter and make a function for each page in ?p, so that the filter only returns the page ID if the (or .. (not .. clause returns true.

8
  • Could you provide some example pages?
    • Some that should be returned.
    • Some that should not be returned.
  • Thinking in terms of functions doesn’t help.
  • All this code is not Clojure, it is Datalog.
    • Clojure (or rather Clojurescript) is used only in the :result-transform and :view parts.
      • With the exception of a limited set of Clojure functions that can be called from within Datalog.
9

Example pages:

File: test concept all headings.md
page-type:: concept

- ## Definition
    - ## Related concepts
- ## Notes

File: test concept no headings.md
page-type:: concept

- blah

File: test concept some headings (missing notes).md
page-type:: concept

- ## Definition
    - ## Related concepts
-

So the above query with an or only returns the page with no headings, where it should also return the page that doesn’t have both headings (the one missing the Notes heading).

10

Try this:

        (or-join [?p]
          (not
            [?notes :block/page ?p]
            [?notes :block/content ?notes-content]
            [(clojure.string/includes? ?notes-content "# Notes")]
          )
          (not
            [?def :block/page ?p]
            [?def :block/content ?def-content]
            [(clojure.string/includes? ?def-content "# Definition")]
          )
        )
11

Thanks again @mentaloid, you’re a legend. Yes, that works.

Couple of questions:

  • Where do I find information about or-join and things like it? It’s not mentioned at all in the official docs. I guess it’s from datalog/datomic? If so, does Logseq use the full spec? (this? Query Reference | Datomic - that seems to have an open ended expression-clause which can be a function, though perhaps I’m not reading it correctly.)
  • Is there a way to programatically extend that or-join, e.g if I want to make one of those header chunks for each header in a provided vector? For this page template, I only have a few headers I need, so it’s easy. Later on, I’m thinking I’ll want some templates with many headers.
12

Unfortunately, the answer is the codebase. Compared to Datomic specs etc., some things are implemented, some are not. Datalog itself is not Turing-complete, so some scenarios are impossible. If you have advanced needs, should sooner or later proceed to custom code (consider kits and the likes).

1 Like