Post processing queries to acheive better sorting

Hi, it appears that several people are having issues with sorting their queries:

Sort next tasks query in reverse order (upcoming -> future)
Query to produce events in chronological order for current page only
Simple Query Sort-By does not work
Advance Query - Sort by latest update datetime
Sorting a query by block order on page?
Separate query result sorting logic from result transform - #2 by douglascamata
Sort deadline query chronologically
Sort NEXT queries by block/journal-day

It seems that there are two main problems

  1. Sorting produces inconsistent results
  2. Formatting and information is lost after a result transform, for me the journal page dates disappearing is the biggest issue.

I have been banging my head against this problem for a long time now and have trialled and errorred my way though 100s of advanced queries trying to solve the issue. I am not a coder and have got nowhere, but I have an idea :bulb:

Logseq databases contain all of their pages as simple text files in the pages and journals directories. I had thought about using a script to run through these files and extract and sort the information I need. However, Logseq’s folder structure sometimes gets nested and complicated. Also, many variables I might like to use in my search like db/id, block/properties etc. might not be available or might be tricky for a script to obtain.

What about this as a solution.
1. First do a query or advanced query from Logseq
1.1 This could be done from a page created only for this purpose so there is not extra information from entries above and below the query
2. Then run a script on the resulting file to put it in the correct order
2.1 I am not capable of doing this myself, but I believe it would not be difficult in bash or python for example.
2.2 Later, this could be made into an Plugin, or perhaps incorporated into an existing plugin such as @adxsoft’s one which works online, or can be found on github. as a Logseq Plugin.

Is this feasible?
If so, can anyone show me how to do it?
Or is there another way to solve the issue?
Either way, I think this function ought to be there and it looks like it would help a lot of people and not just me.

The way I see it, the biggest issue is that using :result-transform in advanced queries overrides the default query output. Sometimes this is a good thing and what the end-user may want. Especially when combined with a :view.

For me there are two main issues.

  • Blocks not staying in the same order when no sorting is applied (Sorting a query by block order on page?)
  • No way to keep the formatting for block output when only wanting to sort the result a certain way
    For the most part I’ve been able to write queries in a way that give me the (sorting) results I want.

Your suggestion about using the files isn’t really an option. Lots of Logseq’s rendering isn’t saved into files.
As you already pointed out, many properties are not in those files, but something Logseq manages in it’s in-memory database.
As for writing a query on a page. Same problem. The query just gets put in plain text in the file, the output is rendered whenever you open that page in Logseq itself.

The way forward is probably the query builder I’ve heard mention of, as well as something to be done about making formatting the query output easier. For example making changes to the :result-transform would add those on top of the default instead of completely overriding them. When they need to be overridden a specific command could be used for that. I think that alone would help a lot.

1 Like

Argh you are right. The files with a query in them, only show the query itself, not the output of the query.
Other plugins like PDF Export Plugin are capable of using the interpreted output however.

…maybe there is hope? :face_with_diagonal_mouth:

2 Likes

Yes, think the solution is to add a sort button within the query builder.

Also, I think some templates which sort items could also be included by default. Sorting is such a powerful feature. (For me, Logseq would lose 50% of its functionality if I wasn’t able to do this). This is also a feature where Logseq is better than a lot of the competition. However, sorting is hugely difficult to do in Clojure for those who are not well acquainted with this language. It took me a lot of mucking around, and trial-and-error to get this to work. Including some templates would really ease the learning curve for a lot of people.

2 Likes