Use queries outputs as an input for Synthesis arithmetic calculations

Customization Synthesis
,
1

Hello. Inspired by this and this, I was wondering if Synthesis would be an appropiate (and relatively simple) solution for the next use case.

Basically, I want to use the output of my basic or advanced queries to compute scalars that provides useful “stats” of my performance. This is a simple example:

  • Let’s say I have pages with property “status:: todo” and other pages with “status:: done”. Can Synthesis compute this expression?

    • var_output = 100*(num_pages_done/num_pages_todo )

    • Where “num_pages_todo” is the number of the pages with “status:: todo” and "num_pages_done " is the number of the pages with “status:: done”.

Or maybe I should to this directly with python scripts?

Thank you.

2
  • A python script could do that, but it would have to parse all the files every time.
    • That would become increasingly slow.
  • Since you already use queries, for your simple example I would suggest some clojurescript, like done in this query:
    #+BEGIN_QUERY
{
 :query [:find (sum ?done) (sum ?todo)
   :keys done todo
   :with ?p
   :where
     [?p :block/properties ?props]
     [(get ?props :status) ?status]
     (or
       (and
         [(= ?status "done")]
         [(* 1 1) ?done]
         [(* 0 0) ?todo]
       )
       (and
         [(= ?status "todo")]
         [(* 0 0) ?done]
         [(* 1 1) ?todo]
       )
     )
     [?p :block/name ?name]
 ]
 :view (fn [result]
   (def r (first result))
   [:div (str "var_output = " (* 100 (/ (:done r) (:todo r) ) ) ) ]
 )
}
#+END_QUERY
  • If that approach doesn’t fit your actual needs:
    • firstly describe them, so as we can suggest the appropriate option
    • there are plenty of other options (Synthesis, kits, plugins etc.)
1 Like
3

Nice. It is a good starting point, thank you. I was not considering clojure because of the complexity, but if it is the best solution for performance, I will focus on it.

Ok, then let me explain the long story. Even though I have a lot of use cases, I will describe the main ones to hopefully converge to the best type of solution for each one and so I can start investigating on it.

  • :dart:Compute percentage of completed goals

    • There are “goal” pages with specific status (todo/going/done/paused)
    • I want to calculate a scalar that represents the progress of the completed goals. That is, the total progress is computed as:
      • total_progress = (done_goal_pages / all_goal_pages) * 100
        • where
          • done_goal_pages are pages with type = goals AND status = done
          • all_goal_pages are all pages with type = goal AND (status = todo OR going)

  • :jigsaw:Compute done goals based on their domain

    • “Goal” pages have a target domain coded in a property called “domain”.
    • For EACH DOMAIN, I want to calculate the achieved goals dedicated to that domain.That is:
      • domain1_done_goals=sum ( (goal pages where (status=done AND domain=domain1) )
      • domain2_done_goals=sum ( (goal pages where (status=done AND domain=domain2) )

  • :chart_with_upwards_trend:Plot XY, table or pie type charts directly on Logseq

    • On the journal pages I fill out my tracking parameters and habits.

    • Lets take a “simple” example: I want to plot the percentage of achieved habits for each day and on a specific date range. That is:

      • Find all the journal pages within specfic dates (from January to February for example)
      • On each journal page within that month, find all blocks with property “type:: habit”
      • For each one of those blocks check the property “value” which contains “1” or “0” depending if I did the routine that day or not.
      • For each day, sum the value property of those blocks and divide them by the number of active habits (say 10 habits) → day_habit percentage = sum (all values of the day / 10)
      • Finally, plot the day_habit percentage variable for each day in the range. Y = variable and X = each day on the Jan-Feb range.

    • I know there are some plugins to achive similar things, like this one or this one but I would like to be “independent” of the plugins for this tracking applications and use my own codes to customize them for the specific use cases.

What do you think would be the best approach for each use case (synthesis, kits, python, even excel/power BI…)??

Thank you for taking the time to read this.

4
  • For queries with minimal calculations, my earlier suggestion stands:
    • Your first case is already covered in my previous answer, if you additionally:
      • add some filtering for type:
        [(get ?props :type) ?type]
[(= ?type "goal")]
      • replace [(= ?status "todo")] with
        (or
  [(= ?status "todo")]
  [(= ?status "going")]
)
    • Your second case looks simpler, as it has:
      • no need for calculations other than (sum ...)
      • a need for some domain filtering
  • For generating graphics, you need specialized code.
    • Therefore, your third case is not addressed at all.
      • Should still use queries to gather the data.
      • Then pass that data to a dedicated library of your preference to generate the plot.
        • From the various options (plugins, external applications etc.), I would suggest matplotlib, in a python-powered kit.
5

Ok, some comments:

  • Use case 1:
    • To understand what you are doing… what are parts like these doing exactly? Assigning 1 and 0 values? If so, why using multiplications like 1 1 and 0 0? Excuse my ignorance.
    • On the other hand, I modified the query and it is working but I realized I made a mistake, because all_goal_pages should equal to ALL the goal pages with status todo, going, paused or done. Not only the todo status. Sorry about the confusion.
      • Any suggestion to redefine the “total_progress” variable taking into account the mentioned modification?
  • Use case 3: Any post, reference or similar information that you recommend to start the matplotlib+python-powered kit solution? I assume this one, right?

Thank you.

6
  • Yes, this is the only way I’m aware of for assigning constant values.
    • equivalent to (+ 1 0), (- 1 1) etc.
  • Apparently, there is no syntax for direct assignment.

Try this:

#+BEGIN_QUERY
{
 :query [:find (sum ?done) (count ?status)
   :keys done all
   :with ?p
   :where
     [?p :block/properties ?props]
     [(get ?props :status) ?status]
     (or
       (and
         [(!= ?status "done")]
         [(* 0 0) ?done]
       )
       (and
         [(= ?status "done")]
         [(* 1 1) ?done]
       )
     )
     [?p :block/name ?name]
 ]
 :view (fn [result]
   (def r (first result))
   [:div (str "var_output = " (* 100 (/ (:done r) (:all r) ) ) ) ]
 )
}
#+END_QUERY

It counts in all every page that has status::. If you want to limit the values, should replace:

  • line [(!= ?status "done")]
  • with [(contains? #{"todo" "going" "paused" ...} ?status)]
7

Understood. Everything is working for me. The only problem I notice is that when no goal status is counted, the query does not display a “0”, but “No matched results”. Is there a way I can initialize
var_output = 0 and then replace the final value in case que query finds the done goals?

imagen → This should be “0”

Thank you!

8

Turning “lack of results” into 0 is kind of arbitrary (may as well return 100 etc.) But if this is what you want, try the following:

#+BEGIN_QUERY
{
 :query [:find (sum ?done) (sum ?all)
   :keys done all
   :with ?p
   :where
     [?p :block/properties ?props]
     (or-join [?all ?done ?props]
       (and
         [(* 1 1) ?all]
         [(get ?props :type) ?type]
         [(= ?type "goal")]
         [(get ?props :status) ?status]
         (or
           (and
             [(!= ?status "done")]
             [(* 0 0) ?done]
           )
           (and
             [(= ?status "done")]
             [(* 1 1) ?done]
           )
         )
       )
       (and
         [(* 0 0) ?all]
         [(* 0 0) ?done]
       )
     )
     [?p :block/name ?name]
 ]
 :view (fn [result]
   (def r (first result))
   (def v (* 100 (/ (:done r) (:all r) ) ) )
   [:div (str "var_output = " (if (= (str v) "NaN") 0 v) ) ]
 )
}
#+END_QUERY

It treats all as a sum, thus lack of goals produces 0/0 (which in Javascript gives NaN), in which case it returns 0. This 0 is hard-coded, so it can be replaced with any other default value.

1 Like
9

For your third case, I would also suggest Charts from queries

1 Like
10

This is nice! I will give it a try asap and give some feedback. Thank you for taking the time for this implementation.