Advanced query for sorted tasks with custom table view

This has been a rough one - I’ve been trying to built a table view of my tasks sorted by urgency and finally got a solution that I’m satisfied with.

As there aren’t many resources on advanced queries (especially custom view funcitons), I decided to share the result. Let me know if you found it useful:

#+BEGIN_QUERY
{:title "BACKLOG"
:query [:find ?status (pull ?todo [*])
  :keys status task
  :where
  [?todo :block/marker ?status]
  [(contains? #{"NOW" "LATER" "DOING" "TODO" "IN-PROGRESS" "WAIT" "WAITING"} ?status)]
  ; I keep all my templates in a page called "templates" and this is how I filter out TODOS defined insde them. You can delete it if you don't have such a page
  (not [?todo :block/page [:block/name "templates"]])]
 :result-transform (fn [result] 
  (sort-by
    (min (fn [d] (get-in d [:task :block/deadline] 99999999)) (fn [d] (get-in d [:task :block/scheduled] 99999999)))
    result
  )
)
:view (fn [rows] 
  (defn dateformat [datestr]
    (let [year (subs datestr 0 4)]
      (let [month (subs datestr 4 6)]
        (let [day (subs datestr 6 8)]
          (str year "-" month "-" day)
        )
      )
    )
  )
  [:div.overflow-x-auto.query-table {:width "100%"} [:table.table-auto 
    [:thead [:tr 
      [:th.whitespace-nowrap ">"] 
      [:th.whitespace-nowrap  {:width "50%"} "Task"] 
      [:th.whitespace-nowrap "Project"] 
      [:th.whitespace-nowrap "Status"] 
      [:th.whitespace-nowrap "Scheduled"] 
      [:th.whitespace-nowrap "Deadline"]
    ]]
    [:tbody (for [r rows] [:tr 
      [:td.whitespace-nowrap [:a {:href (str "#/page/" (get-in r [:task :block/uuid]))} ">" ] ]
      [:td.whitespace-nowrap (clojure.string/replace (first (str/split-lines (get-in r [:task :block/content]))) (re-pattern "^[^ ]+ ") "")]
      [:td.whitespace-nowrap (if (not (nil? (get-in r [:task :block/properties :project]))) [:a {:href (str "#/page/" (first (get-in r [:task :block/properties :project])) )} (str "[[" (first (get-in r [:task :block/properties :project])) "]]") ]) ]
      [:td.whitespace-nowrap (str (get-in r [:task :block/marker])) ]
      [:td.whitespace-nowrap (dateformat (str (get-in r [:task :block/scheduled]))) ]
      [:td.whitespace-nowrap (dateformat (str (get-in r [:task :block/deadline]))) ]
    ]) ]
  ] ]
)
:collapsed? false}
#+END_QUERY

Result:

Full (updated) example:

Great work @theCalcaholic :clap:t2:

Could you please explain the role/function of the following line:

:keys status task

Including :keys in an advanced query changes the structure of the query results. In specific, the result structure is changed in the following ways:

  • structure changes from a list of the data found via the :find expression
  • structure changes into a list of maps. There’s one set of maps for each item found, and each set has the same number of maps as there are :keys.

In the :keyed result, each map gets it’s key name from one of the values after :keys. These values are named symbols. For example, the symbols from OP’s query are status and task.

Let’s peek at the result of the original advanced query to compare the result with and outout :keys. This is possible by changing the :view function to :view :pprint

advanced query

Advanced query used to see the results. Notice the modified :view function.

{:query
 [:find ?status (pull ?todo [*])
  :keys status task ;; comment this out
  :where
  [?todo :block/marker ?status]
  [(contains? #{"NOW" "LATER"
                "DOING" "TODO"
                "IN-PROGRESS" "WAIT"
                "WAITING"}
              ?status)]]
 :result-transform
   ;; same as above
 
 :view :pprint
 }

query results

Comparing the advanced query result both with and without the :keys feature.

Without :keys
This is the “default” query result structure. It doesn’t use :keys status task

(
 "TODO"

 {:block/uuid #uuid "6682d233-653e-47b4-9196-9fd5b19e60c8",
  :block/content "TODO do laundry",
  :db/id 8926,
  :block/marker "TODO"}

 "TODO"
 
 {:block/uuid #uuid "6682d234-ad2f-4584-b660-60c22d1074bd",
  :block/content "TODO canned tomato paste",
  :db/id 9542,
  :block/marker "TODO"}
 )

With :keys
This shows how the structure of the query result changes when including :keys status task. The content is the same, but the data structure is different. Try to notice what changed and what’s the same.

(
 {:status "TODO",
  :task {:block/uuid #uuid "6682d233-653e-47b4-9196-9fd5b19e60c8",
         :block/content "TODO do laundry",
         :db/id 8926,
         :block/marker "TODO"}}

 {:status "TODO",
  :task {:block/uuid #uuid "6682d234-ad2f-4584-b660-60c22d1074bd",
         :block/content "TODO canned tomato paste",
         :db/id 9542,
         :block/marker "TODO"}}
)

Interpretation

By using :keys in an advanced query, instead of a getting…
( "a list of strings" {:and "maps"} ), logseq returns a
( {:list "of nested"} {:maps "and strings"} )

Aside: About data structures
In language clojure, the syntax tells you the data type
"This is a string"
"This is a
    multi-line string"
0 <- number
1.7 <- number
:i-am-a-keyword <- keyword
symbol <- used for variables, evaluates to something else
(1 2 3 4 5) <- parentheses make lists
{:keyword "string"} <- curly braces make maps storing key-value pairs.
{"string" {:another "map"}} <- Maps can contain other data types
Anyway…

In the result with :keys, each component of the :find clause corresponds to the symbol from :keys in the same position. Each :key symbol becomes the map keyword for that :find clause return value.

Annotated results

Without :keys
Query results without using :keys with data type annotations:

(       ;; list data type
 "TODO" ;; string data type   ;; :find ?status
 {      ;; map data type      ;; :find (pull ?todo [*])
  :block/uuid #uuid "6682d233-653e-47b4-9196-9fd5b19e60c8",
  :block/content "TODO do laundry",
  :db/id 8926,
  :block/marker "TODO"}
 
 "TODO"

 {:block/uuid #uuid "6682d234-ad2f-4584-b660-60c22d1074bd",
  :block/content "TODO canned tomato paste",
  :db/id 9542,
  :block/marker "TODO"}
 )

With :keys status task
Query results when using :keys status task:

(  ;; list data type
 { ;; map data type
  :status "TODO",   ;; :find ?status :keys status
  :task {           ;; :find (pull ?todo [*]) :keys task
         :block/uuid #uuid "6682d233-653e-47b4-9196-9fd5b19e60c8",
         :block/content "TODO do laundry",
         :db/id 8926,
         :block/marker "TODO"
        }
  }
 
 {:status "TODO",
  :task {:block/uuid #uuid "6682d234-ad2f-4584-b660-60c22d1074bd",
         :block/content "TODO canned tomato paste",
         :db/id 9542,
         :block/marker "TODO"}
  }
 )

Details

return map

The name for the :keys syntax is a return map in the datomic query language. From the datomic documentation,

Return Maps
Supplying a return-map will cause the query to return maps instead of tuples. Each entry in the :keys / :strs / :syms clause will become a key mapped to the corresponding item in the :find clause.

Return maps also preserve the order of the :find clause.

Datomic grammar
The Datomic EBNF grammar specification shows the relationship between return maps and return-keys. Here’s a simplified version showing the relationship.

Datomic Query Argument Grammar
(simplified)

EBNF rule definition
query [find-spec return-map-spec? with-clause? inputs? where-clauses?]
find-spec ‘:find’ (variable OR pull-expr OR aggregate)
return-map-spec (return-keys OR return-syms OR return-strs)
return-keys ‘:keys’ plain-symbol+
return-syms ‘:syms’ plain-symbol+
return-strs ‘:strs’ plain-symbol+
plain-symbol symbol that does not begin with “$”, “?”, or “%”
with-clause ‘:with’ variable+

Syntax used in grammar

syntax symbol meaning
‘’ literal
() grouping
+ one or more
4 Likes

Wow, thank you for your detailed analysis. Way better than I could’ve put it. Also, I didn’t know about :pprint, I was looking very long for something like that.

To be honest, I wrote part of the query so long ago that I didn’t know completely why I did everything (which is probably why I wasn’t the status key in the view function but :task/:block/marker directly), so your explanation is also very helpful for me as a beginner when it comes to datomic (and the LogSeq APIs available to queries).