# VIEW-005: Data Lineage Panel - NOT PRODUCTION READY

**Status**: ⚠️ **NOT PRODUCTION READY** - Requires significant debugging and rework

**Date Marked**: January 4, 2026

---

## Summary

The Data Lineage panel was developed to trace data flow upstream (sources) and downstream (destinations) through the node graph. While the core engine and UI components were built, the feature has been **disabled from production** due to persistent issues that make it unusable in its current state.

---

## Issues Identified

### 1. **Event Handling / Timing Issues**

- Context menu event fires but panel shows "No node selected"
- Node selection state doesn't propagate correctly to the panel
- Attempted fixes with setTimeout and direct event passing didn't resolve the issue

### 2. **Excessive/Irrelevant Data in Results**

- Simple 3-node connections (Variable → String → Text) show 40+ upstream steps
- All unconnected ports being treated as "sources"
- Signal ports, metadata ports, and visual style properties included in traces
- Results are overwhelming and impossible to understand

### 3. **Filtering Inadequate**

- Port filtering (signals, metadata) only partially effective
- Primary port mapping not working as intended
- Depth limiting (MAX_DEPTH) not preventing noise

---

## What Was Implemented

✅ **Core Engine** (`graphAnalysis/lineage.ts`)

- Upstream/downstream tracing logic
- Component boundary crossing
- Connection resolution

✅ **UI Components**

- `DataLineagePanel.tsx` - Main panel component
- `LineagePath.tsx` - Path display component
- `PathSummary.tsx` - Summary statistics
- React hooks for lineage calculation

✅ **Integration** (Now Disabled)

- ~~Context menu "Show Data Lineage" option~~ (commented out)
- Sidebar panel registration
- EventDispatcher integration
- Canvas highlighting for lineage paths

---

## Attempted Fixes

### Fix Attempt 1: Port Filtering

**Approach**: Filter out signal ports (`changed`, `fetched`) and metadata ports (`name`, `savedValue`)  
**Result**: ❌ Reduced some noise but didn't solve the fundamental issue

### Fix Attempt 2: Skip Unconnected Ports

**Approach**: Don't treat every unconnected input as a "source"  
**Result**: ❌ Should have helped but issue persists

### Fix Attempt 3: Primary Ports Only

**Approach**: Only trace main data port (`value` for Variables, `text` for Text nodes)  
**Result**: ❌ Not effective, still too much data

### Fix Attempt 4: Depth Limiting

**Approach**: Reduced MAX_DEPTH from 50 to 5  
**Result**: ❌ Didn't prevent the proliferation of paths

### Fix Attempt 5: Event Timing

**Approach**: setTimeout wrapper, then removed it  
**Result**: ❌ Neither approach fixed selection state issue

---

## What Needs to be Done

### Critical Issues to Fix

1. **Debug Selection State**

   - Why doesn't the node ID reach the panel?
   - Is the event system working correctly?
   - Add comprehensive logging to trace the full event flow

2. **Rethink Tracing Algorithm**

   - Current approach of "trace all ports unless filtered" is fundamentally flawed
   - Need a "trace only connected ports" approach from the ground up
   - Should only follow actual wire connections, not enumerate ports

3. **Better Port Classification**

   - Distinguish between:
     - **Data ports** (value, text, items)
     - **Style ports** (color, fontSize, padding)
     - **Event ports** (onClick, onHover)
     - **Metadata ports** (name, id)
   - Only trace data ports by default

4. **Smarter Termination Conditions**
   - Don't mark unconnected ports as sources/sinks
   - Only mark actual source nodes (String, Number, etc.) as sources
   - Properly detect end of lineage chains

### Recommended Approach

**Start Fresh with a Focused Scope:**

1. **Phase 1**: Get the basics working for a single use case

   - Simple Variable → connection → Text node
   - Should show exactly 2-3 steps
   - Must display correctly in panel

2. **Phase 2**: Add one complexity at a time

   - Expression nodes (transformation)
   - Component boundaries
   - Multi-hop paths

3. **Phase 3**: Handle edge cases
   - Cycles
   - Multiple sources/destinations
   - Different node types

**Test-Driven Development:**

- Write tests FIRST for each scenario
- Verify traced paths match expectations
- Don't move on until tests pass

---

## Current Code State

The code is **present but disabled**:

### Disabled

- Context menu option (commented out in `nodegrapheditor.ts` lines ~2585-2600)
- Users cannot access the feature

### Still Present

- Panel component (`DataLineagePanel/`)
- Lineage engine (`utils/graphAnalysis/lineage.ts`)
- Sidebar registration (panel exists but hidden)
- All UI styling

**To Re-enable**: Uncomment the context menu section in `nodegrapheditor.ts` after issues are fixed.

---

## Files Involved

### Core Logic

- `packages/noodl-editor/src/editor/src/utils/graphAnalysis/lineage.ts` - Tracing engine
- `packages/noodl-editor/src/editor/src/utils/graphAnalysis/traversal.ts` - Port connections
- `packages/noodl-editor/src/editor/src/utils/graphAnalysis/crossComponent.ts` - Boundary crossing

### UI Components

- `packages/noodl-editor/src/editor/src/views/panels/DataLineagePanel/DataLineagePanel.tsx`
- `packages/noodl-editor/src/editor/src/views/panels/DataLineagePanel/components/LineagePath.tsx`
- `packages/noodl-editor/src/editor/src/views/panels/DataLineagePanel/components/PathSummary.tsx`
- `packages/noodl-editor/src/editor/src/views/panels/DataLineagePanel/hooks/useDataLineage.ts`

### Integration Points

- `packages/noodl-editor/src/editor/src/views/nodegrapheditor.ts` - Context menu (disabled)
- `packages/noodl-editor/src/editor/src/router.setup.ts` - Sidebar routing
- `packages/noodl-editor/src/editor/src/models/sidebar/sidebarmodel.ts` - Panel registration

---

## Conclusion

This feature **needs substantial rework** before it can be production-ready. The current implementation is not salvageable with small fixes - it requires a fundamental rethink of the tracing algorithm and careful test-driven development.

**Estimated Effort**: 2-3 days of focused work with proper debugging and testing

**Priority**: Low - This is a "nice to have" feature, not critical functionality

---

## Related Documentation

- See: `dev-docs/tasks/phase-4-canvas-visualisation-views/CLINE-INSTRUCTIONS.md`
- See: `dev-docs/reference/LEARNINGS.md` for general debugging patterns

---

**Last Updated**: January 4, 2026  
**Marked Not Ready By**: Cline (AI Assistant)  
**Approved By**: Richard Osborne