OpenNoodl/dev-docs/tasks/phase-3-editor-ux-overhaul/TASK-011-advanced-code-editor/TASK-011-PHASE-4-DOCUMENT-STATE-FIX.md

TASK-011 Phase 4: Fix Document State Corruption (Final 5%)

Status: 🟡 Ready to Start
Priority: P1 - High (Editor 95% working, final polish needed)
Started: 2026-01-11
Depends on: TASK-011-PHASE-3 (Completed)

---

Context

Phase 3 successfully fixed the critical cursor positioning and feedback loop issues! The editor is now 95% functional with excellent features:

✅ What's Working Perfectly (Phase 3 Fixes):

• ✅ Syntax highlighting with VSCode Dark+ theme
• ✅ Autocompletion with Noodl-specific completions
• ✅ Linting and inline error display
• ✅ Cursor positioning (FIXED - no more jumps!)
• ✅ Click positioning (accurate)
• ✅ Arrow navigation (smooth)
• ✅ Basic typing (no lag)
• ✅ Format button (Prettier integration)
• ✅ History tracking and restore
• ✅ Resize functionality
• ✅ Keyboard shortcuts (Cmd+S, Cmd+/, etc.)
• ✅ Line numbers, active line highlighting
• ✅ Search/replace
• ✅ Undo/redo

---

🔴 Remaining Issues (5%)

Issue 1: Empty Braces + Enter Key Corruption

Problem: When typing {} and pressing Enter between braces, document state becomes corrupted:

1. Type { → closing } appears automatically ✅
2. Press Enter between braces
3. BUG: Closing brace moves to line 2 (should be line 3)
4. BUG: Left gutter highlights lines 2+ as if "inside braces"
5. Try to type text → each character appears on new line (SEVERE)
6. Fold/unfold the braces → temporarily fixes, but re-breaks on unfold

Expected Behavior:

{
  █  // Cursor here with proper indentation
}

Actual Behavior:

{
}█  // Cursor here, no indentation
// Then each typed character creates a new line

Issue 2: JSON Object Literal Validation

Problem: Typing {"foo": "bar"} shows error: Unexpected token ':'

Needs Investigation:

• This might be correct for Expression validation (objects need parens in expressions)
• Need to verify:
  • Does ({"foo": "bar"}) work without error?
  • Is this only in Expression nodes (correct) or also in Script nodes (wrong)?
  • Should we detect object literals and suggest wrapping in parens?

---

Root Cause Analysis

Issue 1 Root Cause: Race Condition in State Synchronization

The Problem:

const handleChange = useCallback(
  (newValue: string) => {
    isInternalChangeRef.current = true;
    // ... update validation, call onChange ...

    setTimeout(() => {
      isInternalChangeRef.current = false; // ❌ NOT RELIABLE
    }, 0);
  },
  [onChange, validationType]
);

useEffect(() => {
  if (isInternalChangeRef.current) return; // Skip internal changes

  // Sync external value changes
  editorViewRef.current.dispatch({
    changes: {
      /* full document replacement */
    }
  });
}, [value, validationType]);

What Goes Wrong:

1. closeBrackets() auto-adds } → triggers handleChange
2. Sets isInternalChangeRef.current = true
3. Calls parent onChange with "{}"
4. Schedules reset with setTimeout(..., 0)
5. BEFORE setTimeout fires: React re-renders (validation state change)
6. Value sync useEffect sees isInternalChangeRef still true → skips (good!)
7. AFTER setTimeout fires: Flag resets to false
8. Another React render happens (from fold, or validation, or something)
9. Value sync useEffect runs with flag = false
10. Full document replacement → CORRUPTION

Additional Factors:

• indentOnInput() extension might be interfering
• closeBrackets() + custom Enter handler conflict
• foldGutter() operations trigger unexpected re-renders
• Enter key handler may not be firing due to keymap order

---

Solution Strategy

Strategy 1: Eliminate Race Condition (Recommended)

Replace setTimeout with more reliable synchronization:

// Use a counter instead of boolean
const changeGenerationRef = useRef(0);

const handleChange = useCallback(
  (newValue: string) => {
    const generation = ++changeGenerationRef.current;

    // Propagate to parent
    if (onChange) onChange(newValue);

    // NO setTimeout - just track generation
  },
  [onChange]
);

useEffect(() => {
  // Check if this is from our last internal change
  const lastGeneration = lastExternalGenerationRef.current;

  if (changeGenerationRef.current > lastGeneration) {
    // We've had internal changes since last external update
    return;
  }

  // Safe to sync
  lastExternalGenerationRef.current = changeGenerationRef.current;
  // ... sync value
}, [value]);

Strategy 2: Fix Extension Conflicts

Test extensions in isolation:

// Start with MINIMAL extensions
const extensions: Extension[] = [
  javascript(),
  createOpenNoodlTheme(),
  lineNumbers(),
  history(),
  EditorView.lineWrapping,
  customKeybindings(options),
  EditorView.updateListener.of(onChange)
];

// Add back one at a time:
// 1. Test without closeBrackets() - does Enter work?
// 2. Test without indentOnInput() - does Enter work?
// 3. Test without foldGutter() - does Enter work?

Strategy 3: Custom Enter Handler (Already Attempted)

Current implementation not firing - needs to be FIRST in keymap order:

// Move customKeybindings BEFORE other keymaps in extensions array
const extensions: Extension[] = [
  javascript(),
  createOpenNoodlTheme(),

  // ⚠️ KEYBINDINGS MUST BE EARLY
  customKeybindings(options), // Has custom Enter handler

  // Then other extensions that might handle keys
  bracketMatching(),
  closeBrackets()
  // ...
];

---

Implementation Plan

Phase 1: Isolate the Problem (30 minutes)

Goal: Determine which extension causes the corruption

1. Strip down to minimal extensions:

   const extensions: Extension[] = [
     javascript(),
     createOpenNoodlTheme(),
     lineNumbers(),
     history(),
     EditorView.lineWrapping,
     customKeybindings(options),
     onChange ? EditorView.updateListener.of(...) : []
   ];
   

2. Test basic typing:

   • Type {}
   • Press Enter
   • Does it work? If YES → one of the removed extensions is the culprit

3. Add extensions back one by one:

   • Add closeBrackets() → test
   • Add indentOnInput() → test
   • Add foldGutter() → test
   • Add bracketMatching() → test

4. Identify culprit extension(s)

Phase 2: Fix Synchronization Race (1 hour)

Goal: Eliminate the setTimeout-based race condition

1. Implement generation counter approach
2. Test that value sync doesn't corrupt during typing
3. Verify fold/unfold doesn't trigger corruption

Phase 3: Fix Enter Key Handler (30 minutes)

Goal: Custom Enter handler fires reliably

1. Move keybindings earlier in extension order
2. Add logging to confirm handler fires
3. Test brace expansion works correctly

Phase 4: Fix JSON Validation (15 minutes)

Goal: Clarify if this is bug or correct behavior

1. Test in Expression node: ({"foo": "bar"}) - should work
2. Test in Script node: {"foo": "bar"} - should work
3. If Expression requires parens: Add helpful error message or auto-suggestion

Phase 5: Comprehensive Testing (30 minutes)

Run all original test cases:

1. ✅ Basic typing: hello world
2. ✅ Empty braces: {} → Enter → type inside
3. ✅ Navigation: Arrow keys move correctly
4. ✅ Clicking: Cursor appears at click position
5. ✅ JSON: Object literals validate correctly
6. ✅ Multi-line: Complex code structures
7. ✅ Fold/unfold: No corruption
8. ✅ Format: Code reformats correctly
9. ✅ History: Restore previous versions
10. ✅ Resize: Editor resizes smoothly

---

Success Criteria

Must Have:

• Type {}, press Enter, type text → text appears on single line with proper indentation
• No "character per line" corruption
• Fold/unfold braces doesn't cause issues
• All Phase 3 fixes remain working (cursor, navigation, etc.)

Should Have:

• JSON object literals handled correctly (or clear error message)
• Custom Enter handler provides nice brace expansion
• No console errors
• Smooth, responsive typing experience

Nice to Have:

• Auto-indent works intelligently
• Bracket auto-closing works without conflicts
• Code folding available for complex functions

---

Time Budget

Estimated Time: 2-3 hours
Maximum Time: 4 hours before considering alternate approaches

If exceeds 4 hours:

• Consider disabling problematic extensions permanently
• Consider simpler Enter key handling
• Consider removing fold functionality if unsolvable

---

Fallback Options

Option A: Disable Problematic Extensions

If we can't fix the conflicts, disable:

• closeBrackets() - user can type closing braces manually
• foldGutter() - less critical feature
• indentOnInput() - user can use Tab key

Pros: Editor is 100% stable and functional
Cons: Slightly less convenient

Option B: Simplified Enter Handler

Instead of smart brace handling, just handle Enter normally:

// Let default Enter behavior work
// Add one level of indentation when inside braces
// Don't try to auto-expand braces

Option C: Keep Current State

The editor is 95% functional. We could:

• Document the brace issue as known limitation
• Suggest users type closing brace manually first
• Focus on other high-priority tasks

---

Testing Checklist

After implementing fix:

Core Functionality

• Basic typing works smoothly
• Cursor stays in correct position
• Click positioning is accurate
• Arrow key navigation works
• Syntax highlighting displays correctly

Brace Handling (The Fix!)

• Type {} → closes automatically
• Press Enter between braces → creates 3 lines
• Cursor positioned on middle line with indentation
• Type text → appears on that line (NOT new lines)
• Closing brace is on its own line
• No corruption after fold/unfold

Validation

• Invalid code shows error
• Valid code shows green checkmark
• Error messages are helpful
• Object literals handled correctly

Advanced Features

• Format button works
• History restore works
• Cmd+S saves
• Cmd+/ toggles comments
• Resize grip works
• Search/replace works

Edge Cases

• Empty editor → start typing works
• Select all → replace works
• Undo/redo doesn't corrupt
• Multiple nested braces work
• Long lines wrap correctly

---

Notes

What Phase 3 Accomplished

Phase 3 fixed the critical issue - the cursor feedback loop that made the editor unusable. The fixes were:

1. Removed setLocalValue() during typing - eliminated re-render storms
2. Added isInternalChangeRef flag - prevents value sync loops
3. Made CodeMirror single source of truth - cleaner architecture
4. Preserved cursor during external updates - smooth when needed

These changes brought the editor from "completely broken" to "95% excellent".

What Phase 4 Needs to Do

Phase 4 is about polishing the last 5% - fixing edge cases with auto-bracket expansion and Enter key handling. This is much simpler than Phase 3's fundamental architectural fix.

Key Insight

The issue is NOT with our Phase 3 fixes - those work great for normal typing. The issue is conflicts between CodeMirror extensions when handling special keys (Enter) and operations (fold/unfold).

---

References

• Phase 3 Task: TASK-011-PHASE-3-CURSOR-FIXES.md - Background on cursor fixes
• CodeMirror Docs: https://codemirror.net/docs/
• Extension Conflicts: https://codemirror.net/examples/config/
• Keymap Priority: https://codemirror.net/docs/ref/#view.keymap

---

Created: 2026-01-11
Last Updated: 2026-01-11