Advanced queries: rant

Thanks again! I also noticed that the result-transform function does not always give the results that I would expect.
I look forward to your next release :slight_smile:

I was thinking it would be useful if result-transform could have a javascript function as its endpoint, since for many of us Javascript functions would be easier to work with than closure.

Thanks for the valuable info!
I tried to debug variable but didn’t succeed.
Can you please send full query that log variable so I can experiment with?
I probably doing something wrong.
Maybe I’m missing custom view thing.
Thanks :pray:

1 Like

Being proficient in both languages now, I would say one is not inherently better, more logical or more natural than the other.
Both SQL and Datalog are powerful, and have their pro’s and cons.

SQL for a very basic query would look like.

select *
from table
where column = value

A very basic Datalog query can be written as.

{:query [:find (pull ?entity [*])
 :where
 [?entity column value]
] }

In SQL we need to know the table, while in Datalog ?entity represents a variable which Datalog will fill for us based on column and value.
In my opinion this syntax is not inherently better or worse, just different. Both queries basically do the same. Give all attribute for this table/entity where a specific column has a specific value.


For everyone I would recommend learning Datalog by starting with this basic intro tutorial. http://www.learndatalogtoday.org/
And look at the advanced queries documentation, which has been improved and also lists multiple extra resources. (https://docs.logseq.com/#/page/advanced%20queries)

Admittedly some technical affinity is needed to learn this. But there is always simple queries and asking others to help with more complex questions.

2 Likes

Late responder, but here’s my 2c because you’ve touched on something important here that I feel could be expanded on. I’d like to see LogSeq grow in it’s user-base and it’s improved accessibility in learning material.

I think most users come from a beginner software skill level. It all “clicked” for me once I had made an honest effort to learn Clojure and Datalog. Contrary to popular belief, knowledge of data structures are inherently not beginner friendly, but are necessary to realise the full LogSeq potential. I blame my own habitual complancency with all things “Data” onto the accessibility of Excel. Frankly, I don’t know what can be best done in this case, but looking backwards this is what I think could help:

  1. A detailed, embedded, easy-access, textbook-like reference guide (I’m talking 400pg user manual equivalent on top of the existing documentation… any volunteers people?)
  2. Somehow the new-user needs to be introduced to the ideas but also have an impressed appreciation on the required comprehension required to really take advantage of LogSeq.
  3. Holistic Examples. I found that holistic examples, like with graphs that included Advanced Queries, felt lacking or hidden. Sure there were simple examples, but I could not see the tangible results that people seemed to be so enthusiastic about in their graphs. It was too theoretical for my smoothness.

(elaboration below)

I’ve felt the way felt in the first post… up until a few days ago. The thing that changed was that I made a dedicated effort to try and learn clojure and datalog. My background is about 5 years of Mechanical Engineering. Coding experience is limited to Excel VBA. I am still learning and much of what I say is an opinion piece rather than an expert review.

Most users come from a beginner software skill level
I found that the resources for low-level LogSeq are difficult to digest and are spread out over various blogs. It does feel esoteric, which I think is a product to the fact that the content creators using the excellent but sporadic information on the forums / official docs. Big shoutout to the content creators for doing all of this in their own time by the way. I did become very motivated by the enthusiasm presented in videos and these discussion forums, but I didn’t get a tangible grasp on the thing that others were presenting. Content was either too simple (high level and conceptual youtube vides), or too complex (ambigious blogs whose authors evidently come from advanced coding backgrounds). I understand basic coding from uni and work, and given the early stage of LogSeq as an open source project I expect the average new user probably have ‘some’ form of conceptual understanding with coding.

It all “clicked” for me once I had made an honest effort to learn Clojure and Datalog
However in the last week I made a dedicated effort to master Datalog (which is the coding language that Advanced Queries use). Previously I only dabbled with these via templates, but I always struggled to make these queries relevant to my own graph. After studying the basic structure of “datoms” [^1] via many websites and getting to about level 10 of one of my learning resources (https://max-datom.com/) something clicked. I could suddenly comprehend Advanced Queries. I felt I had suddenly developed a sense of intuition into the whole system, much like one has when learning a language and then realising that you can suddenly follow conversation. I began playing around with the developer data which shows the datoms of your LogSeq graph. All of a sudden my graph and my use of it became so much less constrained, and the advanced resources online become user friendly. This leads me to suspect the weakness inherent in newstarters without a background in software…
([^1] Datoms are the data format that logseq database appears to use)

Contrary to popular belief, knowledge of data structures are inherently not beginner friendly, but are necessary to realise the full LogSeq potential
Now I must apologise because I’m a layman laying down big statements (so plug in a big opinionated “I think that…” in front of this paragraph title hahaha) but I think my frustration over available material versus my pace of understanding was because" the LogSeq DataLog framework was so alien to me as a beginner but I was biased to expect simplicity. I grew up under the convenient lens of excel, never questioning the complexity behind the tables, filters, and data relationships. Just bloody pivot table that beach… I think that its this fact that makes most people complacent (myself included) when trying to upgrade from using LogSeq as 1. a notepad and then 2. a data storage and manipulation tool; it’s too big a leap and quite frustrating when the average user thinks their knowledge of data systems is top tier (thanks to their certificate in “Excel - advanced conditional formatting”. What seems an accessible feature (even though it is under the category of advanced in LogSeq documents… damn my ego) is actually equivalent to the study one might expect in the 1st week of a uni course.

So really get the most out of LogSeq and it’s data, one has to try to learn the data structure and syntax behind it.

Frankly, I don’t know what can be best done in this case, but looking backwards this is what I think: (some of these are probably already implemented, but I’m just spitballing.

1. A detailed, embedded, easy-access, textbook-like reference guide for the advanced side of LogSeq data (including datalog and clojure docs).
2. I think it’s important to introduce some of these topics in a very beginner friendly way, and via a contents page, allow the reader to skip ahead. Somehow the new-user needs to be introduced to the ideas but also have an impressed appreciation to the required comprehension required to really take advantage of LogSeq. I think of my textbooks where I’d skip way ahead, have a moment of “nooope”, and then revert to the earlier chapters to try and recapture
3. Holistic Examples. These kinds of things are what really attracted me to LogSeq. Being able to see the way people used LogSeq and Advanced Queries, as well as the enthusiasm behind them. I found that holistic examples, like with graphs that included Advanced Queries, felt lacking or hidden. I could not see the tangible results that people seemed to be so enthusiastic about in their graphs. My failure oftenmost was in my attempt in copying and pasting Advanced Queries and then not getting an expected result due to my lack of understanding.

1 Like

I made mention of it in my previous post. But I have a background in SQL. Like learning it in school in 2004-2008 and using it in my work until literally today :joy:
And honestly… Datalog did not come easy to me, regardless that I have quite a good grip on data structure in a general sense.
But after 3 months I also had a moment it clicked. So I really do agree with you that it is not beginner friendly at all.

Hmmmm you make a good point. Maybe there is a way to use that entry way to explain concepts :thinking:

Yeah there should be a more concise, direct way to learn instead of sending users all over the place. A more Logseq minded way too. We can point at “general” tutorials and then users come back to Logseq with an expectation and then it turns out Logseq works just ever so slightly different.

I was actually having the idea yesterday. I realized that all the resources I offered in the Queries for Task management thread have become… buried in a way. Even when I go back there to find stuff to help people out in other threads I have a hard time finding stuff. And I wrote most of it :sweat_smile:
So I had the idea that some of that was perhaps better shown and explained in a demo graph. Mind you that would be an effort… and I don’t have a lot of spare energy to work on such things. At the same time I think it might be a valuable resource as well.
It’s also one of the reasons I wanted to make a blog series about my own systems and workflows… though it has been… difficult to properly set it up. Even with my boyfriend proofreading and telling me he has no idea what I’m talking about haha.
I’m just not a very good writer of this kind of thing. So I just build advanced queries left and right :smiley: try to add documentation here and there… hope for the best? :stuck_out_tongue:

Thanks for your edition, I really agree with your post.

1 Like

Thanks for your response, I’m glad that some of the key themes of what I said may be able to add something new to this overall conversation.

With regards to your personal efforts within these forums, I hope what I say doesn’t appear to discourage the open dialogue that is within LogSeq resources, forums, or personal blogs; I think it is a very healthy culture of open discussion. I think forum additions such as the ones you make on advanced queries are incredibly valuable. It served me very well and helped turned theory into practice. It’s true, like you say, that posts or blogs may become buried over time, but it’s still good and open information which can be found again. I think and hope that they’ll all eventually serve as building blocks for a consolidated works in the future.

1 Like

It doesn’t for me :slight_smile:
It just makes me think that in my effort to help others, the information just get more spread out.
Many have asked about good resources and all I can do is point them to a link with links, you know?
So I’m just sad I haven’t documented my own learning effort, or been really good at collecting all the help I’ve offered into one place. Now if I wanted to do such a thing it would be retroactive so a lot more work and effort. Something I don’t know if I’ll be able to do.

1 Like

I am a non tech person no coding skills. LogSeq is super useful in my work (researching suicide prevention) but just like Obsidian, Tana and the rest it has been a pain staking experience to learn how to use it.

I think there is a tunnel vision problem with all these PKM tools the presumption that people speak the same advanced language. There is a real lack of onboarding new and novice uses and if these tools are not careful they will remain forever a niche node. If you want these tools to grow and become more “main stream” then these tools need to be taken out of the “secret club” and non tech people like me who would greatly benefit from them considered.

We are only as skilled as the people we can teach

The official logseq instructions and documentation are often gooblygook to me and I have pain staking but been able to learn enough to make the tool half useful but I would like to use more features.

Queries is a big one especially advanced but there are other things that could be made better.
Even many of the plugin instructions are alien to me

Love this tool but as an outsider I do not feel I belong in this club . As a result I find myself looking for other tools and communities where non-techies have some chance of getting the best out of them.

just my rant from an outsiders view

2 Likes

Users like you could be a great asset/help in this process too. Developer could learn from people like you what assumptions they make about their product that is not obvious to their new users.
I know they are working on a better onboarding and I hope this will better provide information for users like you.

2 Likes

Good morning @Siferiax

Thankyou for your response. I do love this app like really love it and I have patched things together over time to some degree. It is so much more superior than Obsidian or Tana for novices like me and that is partially (and positively) due to the almost minimalist design. Its not overwhelming when you first open it (compared to the other two wich is just to much when you do not know much about these things) that in itself is part of the reason I switched to LogSeq so I can imagine and do beleive LogSeq has a massive advantage over the other tools in attracting a less techsavvy audience and as such should really pump and push that. Logseq definately has a better opportunity to pull in an outer ring of people where OBsidian and Tana would scare them off (as in my case). +1 there for Logseq.

When the UK went into its first lockdown I had the time to persevere and patch my knowledge together but if I discovered any of these tools today I would not of had the time to educate myself on these and would of moved on to a simpler (but less useful) note taking app.

So there is definitely a space to focus on simpler learning or bite size learning for novices. It may also be useful if there was a dedicated thread for novices and everyone else could pop in and help us that would be great. Discord is too mad for me and when you are trying to learn something that is hard to you a slower space is needed so this forum is a good, clean, minimal and slower space to learn.

I hope no one thinks I am being negative ninny. I do love this app and can see how useful it is. I can see its potential and I would love to use it to teach people therapeutic journaling (and it really would help people discover themselves) but it needs to come general user friendly for me to be able to do this.

Happy to thrash out ideas on how to make this product more user or user language friendly if I can contribute in any way, shape or form.

One thought I have constantly (but no idea if possible) would be to actually have inbuilt queries for common functions like a drop down box "select this (for example) all todos for the coming week) press hey presto theres your list. Does that make sense?

respectfully

Kane

1 Like