Logseq sync: content got overwritten when edit without fully synced

If a page is edited when not fully synced, the new changes might overwrite the remote contents that not synced to local

  • mostly happens when switched to a new device, launch Logseq, and edit pages without waiting the :cloud::yellow_circle: icon on the top-right to become :cloud::green_circle:.
  • another case is using quick capture on mobile platform and it edit contents without waiting (it’s a bug under fix)

It’s because that Logseq Sync is currently using a “newer first” strategy on the file timestamps. The edit on the new device has a newer timestamp, while the remote content isn’t synced to the new device yet.

To recover the overwritten data, may check these on ALL of your devices:

  1. The page history feature in Logseq
  2. The recycle directory for recent deleted / overwritten files in <graph directory>/logseq/bak
    Also see Data Protection and Recovery in Logseq

We are discussing the sync resolution combining text & db structure on Discord Thread

Also see: I'm using Logseq Sync, what should I do if I am experiencing sync issues?

May 10 2023 update:
Working-in-progress PR for the file resolution integration:

Feb 15 2023 update:
We just have some break-through on sync conflict resolution, and some kinda implicit “universal block uuid preservation” unifying db & text files, while keeping robustness to external (offline) editing. It is a block-granulation resolving.

Hopefully we can ship it to the production environment soon, and the multi-device editing experience & robustness would be significantly improved.

5 Likes

This is a case where logseq must be very careful to manage conflicts.

If local copy is changed before server version has been synced the system MUST recognise this as a conflict and either resolve the conflict automatically (if possible) or alert the user about the conflict and let her solve it herself before continuing.

8 Likes

As an outliner, it seems clear that Logseq should not consider files the atomic unit of updates, but rather items/lines. This is one of the biggest reasons to build a custom sync engine rather than relying on a generic file sync service in the first place.

At minimum I would expect line-level merge; more ideal would be an actual structural diff and merge on the outline data structure (including recognizing when a node has been changed/indented/etc. as a relative edit that correspondingly affects the rest of the subtree, not a raw line-level text diff on the Markdown or Org representation).

9 Likes

Line-level merge (diff / delta) for file storage could potentially corrupt outline structures.
We are developing our DB storage solution: Trello

3 Likes

I was pointed to this thread from the Discord thread about syncing.

I think JRK has exactly the right framing of this problem-solution.

To add on, I find the version history UI basically doesn’t work very well:

  • I often can’t scroll through the histories
  • I wish there was a proper visual diff like in Github, because when there have been many edits I dread having to look through all the versions to try and figure out the difference by myself

@janbaykara Exactly!

@jrk Some updates:

We just have some break-through on sync conflict resolution, and some kinda implicit “universal block uuid preservation” unifying db & text files, while keeping robustness to external (offline) editing. It is a block-granulation resolving.

Hopefully we can ship it to the production environment soon, and the multi-device editing experience & robustness would be significantly improved.

4 Likes

Fantastic to hear. I’m really glad you all are taking this very seriously and from your descriptions here and on Trello it sounds like you very much have your heads on straight about how to do it. :slight_smile: Retaining the ability to allow external edits of the text representation in the filesystem is even better.

A peek on the block-based three-way merge:

2 Likes

My sync issues come from the annoying logout timing on Logseq. The 14 day limit appears to be more random, so this morning i worked on macbook in LS syncing ok. I later noticed my imac version had logged out. When i logged in, the imac journal for today overwrote all the work i did on the macbook.

What is the status of sync? I want to use Logseq across both Android and iOS and therefore iCloud doesn’t work for me and sync sounds ideal. I’ve had issues with data loss on iCloud where I haven’t realised a file has not synced, so it’s worrying that this was also happening here.

I love Logseq so I’m hoping the issues are being resolved :grinning: I appreciate multiple block updates from devices to the same file are complex to resolve.

The Sync will come to formal release instead of the beta status when the conflict resolution is merged and QAed. Enhance Logseq Sync: 2 stage file merge by andelf · Pull Request #9238 · logseq/logseq · GitHub

2 Likes

I just lost a tragic about of work because of sync. Content that had been under continuous edit for hours was not written to disk and then was it was overwritten by sync for no good reason… the remote graph hadn’t been updated for hours and hours… this makes no sense at all as it defeats any file based backup system (like Time Machine.)

So, yeah, beware.

continuous edit for hours was not written to disk

Sync won’t work if Logseq can’t write to disk. Need clue on why it’s not writing to disk. It’s not a common report.

not sure what to say. Time Machine on my mac reflects lots of updates to a lot of files, but no updates to any logseq md files. Logseq could write to disk, it just didn’t. This is a fresh OS install on a computer with >1TB of disk space that has shown no other issues.

Updating to 0.9.10 and enabling the experimental syncing seems to have helped me.

I had lost content that I then found in the /bak directory, and anytime I tried to copy it and put it back where it was supposed to go, Logseq would overwrite that file back to the blank state and page history would show that it never had content to begin with.

I enabled the new feature and once again copied from /bak and now it seems to work. Visually it looked like Logseq tried to delete the content again but then it resolved to normal.

1 Like

Thank you. We are making the smart merge feature more robust and it will become the default behavior soon.

1 Like

I had this issue also, but I’m sure it was written to disk. Because I’m my usage, logseq has never failed to write to disk. I noticed sync was off. I turned it on and lost the ten minutes of writing I’d done. This was with smart sync enabled.

The smart sync won’t be work until the first sync with remote server. So be sure that no conflict editing happens before synced successes when you first set it up.

It was not the first sync. It was just one of those frequent occasions where Logseq turns off its sync by itself for no reason.

is there a way to see the file history on the mobile app? on a computer, if i lose a note due to a syncing issue, i can always recover manually from an old version of the file. the most frustrating to me is losing a note i took on my phone, usually because my computer overwrote it.