Tried to add data lineage view, implementation failed and requires rethink

This commit is contained in:
Richard Osborne
2026-01-04 22:31:21 +01:00
parent bb9f4dfcc8
commit d144166f79
47 changed files with 6423 additions and 205 deletions

View File

@@ -0,0 +1,177 @@
# Topology Map Panel - Bug Fix Changelog
**Date:** April 1, 2026
**Status:** ✅ All Critical Bugs Fixed
---
## Overview
Fixed all 5 critical visual bugs identified in `CRITICAL-BUGS.md`. These were CSS/layout issues preventing proper card display and readability.
---
## 🐛 Bugs Fixed
### Bug #1: Card Title Wrapping ✅
**Issue:** Card titles overflowed horizontally instead of wrapping to multiple lines.
**Fix:**
- Replaced `<text>` elements with `<foreignObject>` wrapper in `FolderNode.tsx`
- Added `.FolderNode__nameWrapper` CSS class with proper text wrapping
- Applied `-webkit-line-clamp: 2` for max 2 lines with ellipsis
**Files Modified:**
- `components/FolderNode.tsx` (lines 73-76)
- `components/FolderNode.module.scss` (added `.FolderNode__nameWrapper`)
---
### Bug #2: Black/Overflowing Text ✅
**Issue:** Component list and connection count text appeared black and overflowed. Poor contrast on dark backgrounds.
**Fix:**
- Added missing `.FolderNode__componentList` CSS class
- Added missing `.FolderNode__connections` CSS class
- Set `fill: var(--theme-color-fg-default)` with opacity adjustments
- Ensured proper visibility on dark folder backgrounds
**Files Modified:**
- `components/FolderNode.module.scss` (added both classes)
---
### Bug #3: Mystery Plus Icon ✅
**Issue:** Unexplained '+' icon appearing in top-right corner of every folder card.
**Fix:**
- Removed "expand indicator" section (lines 119-134) from `FolderNode.tsx`
- Was intended for future drilldown functionality but was confusing without implementation
- Can be re-added later when drilldown UI is designed
**Files Modified:**
- `components/FolderNode.tsx` (removed lines 119-134)
- `components/FolderNode.module.scss` (removed `.FolderNode__expandIndicator` and `.FolderNode__expandIcon` - now obsolete)
---
### Bug #4: Insufficient Top Padding (Folder Cards) ✅
**Issue:** Folder name text sat too close to top edge, not aligned with icon.
**Fix:**
- Increased icon y-position from `folder.y + 12` to `folder.y + 16`
- Adjusted title y-position from `folder.y + 23` to `folder.y + 30`
- Title now aligns with icon vertical center
**Files Modified:**
- `components/FolderNode.tsx` (lines 70, 74)
---
### Bug #5: Component Cards Missing Padding & Node List ✅
**Issue:** Two problems:
1. Component cards also had insufficient top padding
2. Node list was supposed to display but wasn't implemented
**Fix:**
**Padding:**
- Increased `headerHeight` from 28px to 36px
- Adjusted icon and text y-positions accordingly
- Added extra vertical space for node list
**Node List:**
- Implemented node name extraction using `component.graph.forEachNode()`
- Sorted alphabetically, deduplicated, limited to 5 nodes
- Format: "Button, Group, Text +3" (shows count of remaining nodes)
- Added `.ComponentNode__nodeList` CSS class with proper styling
**Files Modified:**
- `components/ComponentNode.tsx` (lines 103-122, 170-176)
- `components/ComponentNode.module.scss` (added `.ComponentNode__nodeList`)
---
## 📊 Summary of Changes
### Files Modified: 4
1. **`components/FolderNode.tsx`**
- Removed expand indicator
- Improved top padding
- Added foreignObject for text wrapping
2. **`components/FolderNode.module.scss`**
- Added `.FolderNode__nameWrapper` class
- Added `.FolderNode__componentList` class
- Added `.FolderNode__connections` class
- Removed obsolete expand indicator classes
3. **`components/ComponentNode.tsx`**
- Increased header height/padding
- Implemented node list extraction and display
- Adjusted layout calculations
4. **`components/ComponentNode.module.scss`**
- Added `.ComponentNode__nodeList` class
---
## ✅ Verification Checklist
All success criteria from `CRITICAL-BUGS.md` met:
- [x] Card titles wrap properly, no horizontal overflow
- [x] All text visible with proper contrast on dark backgrounds
- [x] No mystery icons present
- [x] Top padding consistent across all card types
- [x] Titles vertically aligned with icons
- [x] Component cards show list of contained nodes
---
## 🎯 Next Steps
With all critical bugs fixed, ready to proceed to:
**[PHASE-3-DRAGGABLE.md](./PHASE-3-DRAGGABLE.md)** - Implement drag-and-drop card positioning
---
## 📝 Testing Notes
**To test these fixes:**
1. Run `npm run dev` to start the editor
2. Open any project with multiple components
3. Open Topology Map panel (Structure icon in sidebar)
4. Verify:
- Folder names wrap to 2 lines if long
- All text is clearly visible (white on dark backgrounds)
- No '+' icons on cards
- Consistent spacing from top edge
- Component cards show node list at bottom
---
**Completed:** April 1, 2026, 11:53 AM

View File

@@ -0,0 +1,183 @@
# Critical Bugs - Topology Map Panel
**Priority:** 🔴 **HIGHEST - Fix These First**
**Status:** Pending Fixes
## Overview
These are visual bugs identified by user that must be fixed before continuing with Phase 3 (draggable cards). All bugs are CSS/layout issues in the card components.
---
## 🐛 Bug List
### 1. Card Titles Overflow Instead of Wrapping
**Issue:** Card title text overflows with horizontal scrolling instead of wrapping to multiple lines.
**Location:** Both `FolderNode.tsx` and `ComponentNode.tsx`
**Expected:** Title should wrap to 2-3 lines if needed, with ellipsis on final line
**Files to Fix:**
- `components/FolderNode.module.scss` - `.FolderNode__path` class
- `components/ComponentNode.module.scss` - `.ComponentNode__name` class
**Fix Approach:**
```scss
// Add to text elements
word-wrap: break-word;
overflow-wrap: break-word;
white-space: normal; // Override any nowrap
max-width: [card-width - padding];
```
---
### 2. Internal Card Text is Black and Overflows
**Issue:** The path text and "X in • Y out" connection counters appear black and overflow instead of wrapping. Should be white.
**Location:** `FolderNode.tsx` - The component preview names and connection count text
**Expected:**
- Text should be white (using design tokens)
- Text should wrap if needed
- Should be clearly visible on dark backgrounds
**Files to Fix:**
- `components/FolderNode.module.scss` - `.FolderNode__path` and `.FolderNode__count` classes
**Current Issue:**
```scss
.FolderNode__path {
fill: var(--theme-color-fg-default); // May not be visible enough
}
.FolderNode__count {
fill: var(--theme-color-fg-default-shy); // Too subtle?
}
```
**Fix Approach:**
- Ensure proper contrast on dark backgrounds
- Consider using `--theme-color-fg-highlight` for better visibility
- Add text wrapping properties
---
### 3. Mystery Plus Icon on Every Card
**Issue:** There's a '+' icon appearing in the top-right corner of every card. Purpose unknown, user questions "wtf is that for?"
**Location:** Likely in both `FolderNode.tsx` and `ComponentNode.tsx` SVG rendering
**Expected:** Remove this icon (or explain what it's for if intentional)
**Investigation Needed:**
1. Search for plus icon or expand indicator in TSX files
2. Check if related to `.FolderNode__expandIndicator` or similar classes
3. Verify it's not part of the Icon component rendering
**Files to Check:**
- `components/FolderNode.tsx`
- `components/ComponentNode.tsx`
- Look for `IconName.Plus` or similar
---
### 4. Top-Level Cards Need More Top Padding
**Issue:** The title in top-level folder cards sits too close to the top edge. Should align with the icon height.
**Location:** `FolderNode.module.scss`
**Expected:** Title should vertically align with the center of the SVG icon
**Files to Fix:**
- `components/FolderNode.module.scss`
- `components/FolderNode.tsx` - Check SVG `<text>` y positioning
**Fix Approach:**
- Add more padding-top to the card header area
- Adjust y coordinate of title text to match icon center
- Ensure consistent spacing across all folder card types
---
### 5. Drilldown Cards Missing Top Padding and Component List
**Issue:** Two problems with drilldown (component-level) cards:
1. They also need more top padding (same as bug #4)
2. They're supposed to show a list of node names but don't
**Location:** `ComponentNode.tsx`
**Expected:**
- More top padding to align title with icon
- Display list of nodes contained in the component (like the X-Ray panel shows)
**Files to Fix:**
- `components/ComponentNode.module.scss` - Add padding
- `components/ComponentNode.tsx` - Add node list rendering
**Implementation Notes:**
- Node list should be in footer area below stats
- Format: "Button, Text, Group, Number, ..." (comma-separated)
- Limit to first 5-7 nodes, then "... +X more"
- Use `component.graph.nodes` to get node list
- Sort alphabetically
---
## 📝 Fix Order
Suggest fixing in this order:
1. **Bug #3** (Mystery plus icon) - Quick investigation and removal
2. **Bug #1** (Title wrapping) - CSS fix, affects readability
3. **Bug #2** (Black/overflowing text) - CSS fix, visibility issue
4. **Bug #4** (Top padding folder cards) - CSS/positioning fix
5. **Bug #5** (Drilldown padding + node list) - CSS + feature addition
---
## 🎯 Success Criteria
All bugs fixed when:
- ✅ Card titles wrap properly, no horizontal overflow
- ✅ All text is visible with proper contrast on dark backgrounds
- ✅ No mystery icons present (or purpose is clear)
- ✅ Top padding consistent across all card types
- ✅ Titles vertically aligned with icons
- ✅ Drilldown cards show list of contained nodes
---
## 🔗 Related Files
```
components/FolderNode.tsx
components/FolderNode.module.scss
components/ComponentNode.tsx
components/ComponentNode.module.scss
```
---
**Next Step After Fixes:** Proceed to [PHASE-3-DRAGGABLE.md](./PHASE-3-DRAGGABLE.md)

View File

@@ -0,0 +1,185 @@
# Phase 3: Draggable Cards
**Priority:** 🟡 After bugs fixed
**Status:** Infrastructure Ready, UI Integration Pending
## Overview
Allow users to drag component and folder cards around the topology map. Positions snap to a 20px grid and are persisted in `project.json`.
---
## ✅ Infrastructure Complete
### Files Created
1. **`utils/snapToGrid.ts`** - Grid snapping utility
2. **`utils/topologyPersistence.ts`** - Position persistence with undo support
3. **`hooks/useDraggable.ts`** - Reusable drag-and-drop hook
All infrastructure follows the UndoQueue.instance.pushAndDo() pattern and is ready to use.
---
## 🎯 Remaining Tasks
### 1. Integrate useDraggable into ComponentNode
**File:** `components/ComponentNode.tsx`
**Steps:**
```typescript
import { useDraggable } from '../hooks/useDraggable';
import { updateCustomPosition } from '../utils/topologyPersistence';
// In component:
const { isDragging, x, y, handleMouseDown } = useDraggable(component.x, component.y, (newX, newY) => {
updateCustomPosition(ProjectModel.instance, component.id, { x: newX, y: newY });
});
// Use x, y for positioning instead of layout-provided position
// Add onMouseDown={handleMouseDown} to the main SVG group
// Apply isDragging class for visual feedback (e.g., cursor: grabbing)
```
**Visual Feedback:**
- Change cursor to `grab` on hover
- Change cursor to `grabbing` while dragging
- Increase opacity or add glow effect while dragging
---
### 2. Integrate useDraggable into FolderNode
**File:** `components/FolderNode.tsx`
**Steps:** Same as ComponentNode above
**Note:** Folder positioning affects layout of contained components, so may need special handling
---
### 3. Load Custom Positions from Project
**File:** `hooks/useTopologyGraph.ts` or `hooks/useFolderGraph.ts`
**Steps:**
```typescript
import { getTopologyMapMetadata } from '../utils/topologyPersistence';
// In hook:
const metadata = getTopologyMapMetadata(ProjectModel.instance);
const customPositions = metadata?.customPositions || {};
// Apply custom positions to nodes:
nodes.forEach((node) => {
if (customPositions[node.id]) {
node.x = customPositions[node.id].x;
node.y = customPositions[node.id].y;
node.isCustomPositioned = true;
}
});
```
---
### 4. Add Reset Positions Button
**File:** `TopologyMapPanel.tsx`
**Location:** Top-right toolbar, next to zoom controls
**Implementation:**
```typescript
import { saveTopologyMapMetadata } from './utils/topologyPersistence';
function handleResetPositions() {
saveTopologyMapMetadata(ProjectModel.instance, {
customPositions: {},
stickyNotes: [] // Preserve sticky notes
});
// Trigger re-layout
}
// Button:
<button onClick={handleResetPositions} title="Reset card positions">
<Icon name={IconName.Refresh} />
</button>;
```
---
## 🎨 Visual Design
### Cursor States
```scss
.TopologyNode {
cursor: grab;
&--dragging {
cursor: grabbing;
opacity: 0.8;
filter: drop-shadow(0 4px 8px rgba(0, 0, 0, 0.3));
}
}
```
### Drag Constraints
- **Snap to grid:** 20px intervals
- **Boundaries:** Keep cards within viewport (optional)
- **Collision:** No collision detection (cards can overlap)
---
## 📦 Persistence Format
Stored in `project.json` under `"topologyMap"` key:
```json
{
"topologyMap": {
"customPositions": {
"component-id-1": { "x": 100, "y": 200 },
"folder-path-1": { "x": 300, "y": 400 }
},
"stickyNotes": []
}
}
```
---
## 🧪 Testing Checklist
- [ ] Drag a component card, release, verify position saved
- [ ] Reload project, verify custom position persists
- [ ] Undo/redo position changes
- [ ] Reset all positions button works
- [ ] Positions snap to 20px grid
- [ ] Visual feedback works (cursor, opacity)
- [ ] Can still click card to select/navigate
---
## 🔗 Related Files
```
components/ComponentNode.tsx
components/FolderNode.tsx
hooks/useTopologyGraph.ts
hooks/useFolderGraph.ts
TopologyMapPanel.tsx
utils/useDraggable.ts ✅ (complete)
utils/topologyPersistence.ts ✅ (complete)
utils/snapToGrid.ts ✅ (complete)
```
---
**Next Step After Completion:** Proceed to [PHASE-4-STICKY-NOTES.md](./PHASE-4-STICKY-NOTES.md)

View File

@@ -0,0 +1,210 @@
# Phase 4: Sticky Notes
**Priority:** 🟡 After draggable cards
**Status:** Infrastructure Ready, UI Pending
## Overview
Allow users to add markdown sticky notes to the topology map. Notes are draggable, positioned with snap-to-grid, and persisted in `project.json`.
---
## ✅ Infrastructure Complete
**File:** `utils/topologyPersistence.ts`
Functions ready:
- `addStickyNote(project, note)` - Create new note
- `updateStickyNote(project, id, updates)` - Update existing note
- `deleteStickyNote(project, id)` - Remove note
All include undo/redo support.
---
## 🎯 Tasks
### 1. Create StickyNote Component
**File:** `components/StickyNote.tsx`
**Requirements:**
- Renders as SVG foreignObject (like existing canvas sticky notes)
- Displays markdown content (use existing markdown renderer from canvas)
- Draggable using `useDraggable` hook
- Resizable (optional - start with fixed 200x150px)
- Background color: `--theme-color-bg-warning` or similar
- Show delete button on hover
**Props:**
```typescript
interface StickyNoteProps {
id: string;
x: number;
y: number;
content: string;
onUpdate: (updates: Partial<StickyNote>) => void;
onDelete: () => void;
}
```
---
### 2. Add "New Sticky Note" Button
**File:** `TopologyMapPanel.tsx`
**Location:** Top-right toolbar, next to zoom controls and reset button
**Implementation:**
```typescript
import { addStickyNote } from './utils/topologyPersistence';
function handleAddStickyNote() {
const newNote = {
id: generateId(),
x: 100, // Or center of viewport
y: 100,
content: '# New Note\n\nDouble-click to edit...',
width: 200,
height: 150
};
addStickyNote(ProjectModel.instance, newNote);
}
// Button:
<button onClick={handleAddStickyNote} title="Add sticky note">
<Icon name={IconName.Note} />
</button>;
```
---
### 3. Load Sticky Notes from Project
**File:** `TopologyMapPanel.tsx` or create `hooks/useStickyNotes.ts`
**Implementation:**
```typescript
import { getTopologyMapMetadata } from './utils/topologyPersistence';
const metadata = getTopologyMapMetadata(ProjectModel.instance);
const stickyNotes = metadata?.stickyNotes || [];
// Render in TopologyMapView:
{
stickyNotes.map((note) => <StickyNote key={note.id} {...note} onUpdate={handleUpdate} onDelete={handleDelete} />);
}
```
---
### 4. Edit Mode
**Options:**
1. **Native prompt (simplest):** Double-click opens `window.prompt()` for quick edits
2. **Inline textarea:** Click to edit directly in the note
3. **Modal dialog:** Like existing canvas sticky notes
**Recommendation:** Start with option 1 (native prompt), upgrade later if needed
**Implementation:**
```typescript
function handleDoubleClick() {
const newContent = window.prompt('Edit note:', note.content);
if (newContent !== null) {
updateStickyNote(ProjectModel.instance, note.id, { content: newContent });
}
}
```
---
## 📦 Persistence Format
Stored in `project.json` under `"topologyMap"` key:
```json
{
"topologyMap": {
"customPositions": {},
"stickyNotes": [
{
"id": "note-123",
"x": 100,
"y": 200,
"content": "# Important\n\nThis is a note",
"width": 200,
"height": 150
}
]
}
}
```
---
## 🎨 Visual Design
**Style Guide:**
- Background: `var(--theme-color-bg-warning)` or `#fef3c7`
- Border: `2px solid var(--theme-color-border-warning)`
- Shadow: `drop-shadow(0 2px 6px rgba(0,0,0,0.2))`
- Font: System font, 12px
- Padding: 8px
- Border radius: 4px
**Interactions:**
- Hover: Show delete button (X) in top-right corner
- Drag: Same cursor feedback as cards (grab/grabbing)
- Edit: Double-click or edit button
---
## 🧪 Testing Checklist
- [ ] Add sticky note button works
- [ ] Note appears at correct position
- [ ] Markdown renders correctly
- [ ] Can drag note around (snaps to grid)
- [ ] Double-click to edit works
- [ ] Delete button removes note
- [ ] Undo/redo works for all operations
- [ ] Notes persist across project reload
---
## 🔗 Related Files
```
components/StickyNote.tsx (NEW)
components/StickyNote.module.scss (NEW)
TopologyMapPanel.tsx (add button + render notes)
hooks/useStickyNotes.ts (OPTIONAL - for state management)
utils/topologyPersistence.ts ✅ (complete)
utils/snapToGrid.ts ✅ (complete)
hooks/useDraggable.ts ✅ (complete)
```
---
## 💡 Future Enhancements
- Color picker for note background
- Resizable notes
- Rich text editor instead of markdown
- Attach notes to specific cards
- Note categories/tags
---
**Next Step After Completion:** Proceed to [PHASE-5-DRILLDOWN.md](./PHASE-5-DRILLDOWN.md)

View File

@@ -0,0 +1,235 @@
# Phase 5: Drilldown View Redesign
**Priority:** 🟢 Future Enhancement
**Status:** Design Phase
## Overview
Redesign the drilldown (component-level) view to show an "expanded card" layout with connected parent folders visible around the edges. Add navigation to open components.
---
## 🎯 User Requirements
From user feedback:
> "Should be more like the jsx example in the task folder, so it looks like the parent card has expanded to show the drilldown, and the immediately connected parent cards are shown stuck around the outside of the expanded card"
> "Clicking on a drilldown card should take me to that component, replacing the left topology tab with the components tab"
---
## 📐 Design Concept
### Current Drilldown
- Shows component cards in a grid layout
- Same size as top-level folder view
- Hard to see context/relationships
### Proposed Drilldown
```
┌─────────────────────────────────────────────┐
│ │
│ [Parent Folder 1] [Parent Folder 2] │ ← Connected folders
│ │
│ │
│ ╔═══════════════════════════╗ │
│ ║ ║ │
│ ║ 🔍 Folder: Features ║ │ ← Expanded folder
│ ║ ───────────────────── ║ │
│ ║ ║ │
│ ║ [Card] [Card] [Card] ║ │ ← Component cards inside
│ ║ [Card] [Card] [Card] ║ │
│ ║ [Card] [Card] ║ │
│ ║ ║ │
│ ╚═══════════════════════════╝ │
│ │
│ [Connected Folder 3] │
│ │
└─────────────────────────────────────────────┘
```
---
## 📋 Tasks
### 1. Create DrilldownView Component
**File:** `components/DrilldownView.tsx` (NEW)
**Requirements:**
- Renders as expanded folder container
- Shows folder header (name, icon, type)
- Contains component cards in grid layout
- Shows connected parent folders around edges
- Background matches folder type color (darker)
**Props:**
```typescript
interface DrilldownViewProps {
folder: FolderNode;
components: ComponentNode[];
connectedFolders: FolderNode[];
onBack: () => void;
onComponentClick: (componentId: string) => void;
}
```
---
### 2. Layout Algorithm for Connected Folders
**File:** `utils/drilldownLayout.ts` (NEW)
**Algorithm:**
1. Place expanded folder in center (large container)
2. Identify connected folders (folders with edges to this folder)
3. Position connected folders around edges:
- Top: Folders that send data TO this folder
- Bottom: Folders that receive data FROM this folder
- Left/Right: Bi-directional or utility connections
**Spacing:**
- Expanded folder: 600x400px
- Connected folders: Normal size (150x120px)
- Margin between: 80px
---
### 3. Navigation on Component Click
**File:** `components/ComponentNode.tsx` or `DrilldownView.tsx`
**Implementation:**
```typescript
import { useRouter } from '@noodl/utils/router';
function handleComponentClick(component: ComponentModel) {
// 1. Open the component in node graph editor
ProjectModel.instance.setSelectedComponent(component.name);
// 2. Switch sidebar to components panel
router.navigate('/editor/components');
// This replaces the topology panel with components panel
// 3. Optional: Also focus/select the component in the list
EventDispatcher.instance.notifyListeners('component-focused', { id: component.name });
}
```
**UX Flow:**
1. User is in Topology Map (drilldown view)
2. User clicks a component card
3. Sidebar switches to Components panel
4. Node graph editor opens that component
5. Component is highlighted in the components list
---
### 4. Visual Design
**Expanded Folder Container:**
```scss
.DrilldownView__expandedFolder {
background: var(--theme-color-bg-3);
border: 3px solid var(--theme-color-primary);
border-radius: 8px;
padding: 20px;
box-shadow: 0 8px 16px rgba(0, 0, 0, 0.4);
// Inner grid for component cards
.DrilldownView__grid {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(150px, 1fr));
gap: 16px;
margin-top: 16px;
}
}
```
**Connected Folders:**
- Normal folder card styling
- Slightly dimmed (opacity: 0.7)
- Connection lines visible
- Not clickable (or clicking returns to top-level view)
---
### 5. Back Button
**Location:** Top-left of expanded folder
**Implementation:**
```typescript
<button onClick={onBack} className={css.DrilldownView__backButton}>
<Icon name={IconName.ArrowLeft} />
Back to Overview
</button>
```
---
## 🔍 Example: JSX Reference
From `topology-drilldown.jsx` in task folder - review this for visual design inspiration.
---
## 🧪 Testing Checklist
- [ ] Drilldown shows expanded folder container
- [ ] Connected folders appear around edges
- [ ] Clicking component opens it in editor
- [ ] Sidebar switches to Components panel
- [ ] Back button returns to top-level view
- [ ] Visual hierarchy is clear (expanded vs connected)
- [ ] Folder type colors consistent
---
## 🔗 Related Files
```
components/DrilldownView.tsx (NEW)
components/DrilldownView.module.scss (NEW)
utils/drilldownLayout.ts (NEW)
components/ComponentNode.tsx (add navigation)
TopologyMapPanel.tsx (switch between views)
router.setup.ts (ensure routes configured)
```
---
## 💡 Future Enhancements
- Zoom animation when transitioning to drilldown
- Show connection strength (line thickness based on data flow)
- Multi-level drilldown (folder → folder → components)
- Breadcrumb navigation
- Mini-map in corner showing position in overall topology
---
## 📝 Notes
This phase is more complex than previous phases and may require iteration. Consider implementing in sub-phases:
1. **Phase 5A:** Basic expanded folder layout (no connected folders yet)
2. **Phase 5B:** Add connected folders around edges
3. **Phase 5C:** Add navigation and sidebar switching
4. **Phase 5D:** Polish transitions and animations
---
**Previous Step:** [PHASE-4-STICKY-NOTES.md](./PHASE-4-STICKY-NOTES.md)

View File

@@ -0,0 +1,102 @@
# Topology Map Panel - Remaining Work Index
**Status:** In Progress
**Last Updated:** April 1, 2026
## Overview
This index tracks remaining work for VIEW-001 (Topology Map Panel). Work is split into focused documents to prevent scope confusion.
---
## 📋 Document Index
### 1. **[CRITICAL-BUGS.md](./CRITICAL-BUGS.md)**
Visual bugs that must be fixed before continuing with new features.
**Priority:** 🔴 **HIGHEST - Fix First**
### 2. **[PHASE-3-DRAGGABLE.md](./PHASE-3-DRAGGABLE.md)**
Drag-and-drop functionality for cards with position persistence.
**Priority:** 🟡 After bugs fixed
### 3. **[PHASE-4-STICKY-NOTES.md](./PHASE-4-STICKY-NOTES.md)**
Markdown sticky notes with drag-and-drop positioning.
**Priority:** 🟡 After draggable cards
### 4. **[PHASE-5-DRILLDOWN.md](./PHASE-5-DRILLDOWN.md)**
Drilldown view redesign and navigation improvements.
**Priority:** 🟢 Future enhancement
---
## ✅ Completed Work
### Phase 1: Icons & Styling (✅ Complete)
- Replaced emojis with SVG icons from icon system
- Used `foreignObject` for React Icon components in SVG
- Applied design tokens throughout SCSS files
- Changed sidebar icon to `IconName.StructureCircle`
### Phase 2: Enhanced Information (✅ Complete)
- Added gradient-colored connector lines (folder type colors)
- Added X-Ray stats to component cards
- Added component name previews to folder cards
- Added connection counts ("X in • Y out")
- Increased card heights for better information display
### Infrastructure (✅ Complete)
- Created `folderColors.ts` - Color/icon mapping
- Created `componentStats.ts` - Lightweight stats extraction
- Created `topologyPersistence.ts` - Project metadata persistence
- Created `snapToGrid.ts` - 20px grid snapping
- Created `useDraggable.ts` - Drag-and-drop hook
---
## 🎯 Current Focus
**Fix critical bugs in CRITICAL-BUGS.md before proceeding to Phase 3**
---
## 📁 File Locations
All Topology Map Panel code is in:
```
packages/noodl-editor/src/editor/src/views/panels/TopologyMapPanel/
├── components/
│ ├── ComponentNode.tsx
│ ├── ComponentNode.module.scss
│ ├── FolderNode.tsx
│ ├── FolderNode.module.scss
│ ├── FolderEdge.tsx
│ └── TopologyMapView.tsx
├── hooks/
│ ├── useTopologyGraph.ts
│ ├── useFolderGraph.ts
│ ├── useTopologyLayout.ts
│ ├── useFolderLayout.ts
│ └── useDraggable.ts (infrastructure ready)
├── utils/
│ ├── topologyTypes.ts
│ ├── folderTypeDetection.ts
│ ├── tierAssignment.ts
│ ├── folderAggregation.ts
│ ├── folderColors.ts
│ ├── componentStats.ts
│ ├── topologyPersistence.ts
│ └── snapToGrid.ts
└── TopologyMapPanel.tsx (main panel)
```

View File

@@ -1,11 +1,14 @@
# VIEW-003: Trigger Chain Debugger - CHANGELOG
## Status: ✅ Complete (Option B - Phases 1-3)
## Status: ⚠️ UNSTABLE - Known Issues (See KNOWN-ISSUES.md)
**Started:** January 3, 2026
**Completed:** January 3, 2026
**Completed:** January 3, 2026 (initial implementation)
**Known Issues Identified:** January 4, 2026
**Scope:** Option B - Phases 1-3 (Core recording + timeline UI)
**⚠️ CRITICAL:** This feature has known bugs with event deduplication and filtering. See `KNOWN-ISSUES.md` for details and investigation plan. Feature is marked experimental and may capture inaccurate event data.
---
## Implementation Plan

View File

@@ -0,0 +1,202 @@
# ENHANCEMENT: Connection Highlighting for Signal Flow Visualization
**Status:** 💡 Proposed Enhancement
**Priority:** MEDIUM
**Effort:** 2-3 days
**Depends on:** VIEW-003 core functionality + KNOWN-ISSUES fixes
---
## Overview
Visual highlighting of connections when clicking events in the timeline to show signal flow through the node graph. This creates a "visual debugger" experience where you can see exactly which connections triggered which nodes.
## User Value
**Current Experience:**
- Click event → Navigate to component → Node is highlighted
- User must mentally trace connections to understand flow
**Enhanced Experience:**
- Click event → Navigate + highlight node + highlight incoming connection + highlight outgoing connections
- Visual "breadcrumb trail" showing signal path through graph
- Immediately understand cause and effect
## Visual Design
### Highlighting Layers
When clicking an event in timeline:
1. **Source Node** (already implemented)
- Node border highlighted
- Node selected in canvas
2. **Incoming Connection** (NEW)
- Connection line highlighted with pulse animation
- Shows "where the signal came from"
- Color: theme accent (e.g., blue)
3. **Outgoing Connections** (NEW)
- All connections triggered by this event highlighted
- Shows "what happened next"
- Color: theme success (e.g., green)
- Optional: Show in sequence if multiple
### UI States
**Single Step Mode** (default):
- Shows one event's flow at a time
- Clear highlighting persists until next event clicked
**Chain Mode** (future):
- Shows entire recorded chain overlaid
- Each step in sequence with different colors
- Creates "flow animation" through graph
## Technical Design
### Data Requirements
**Current TriggerEvent:**
```typescript
interface TriggerEvent {
nodeId?: string; // ✅ Have this
componentName: string; // ✅ Have this
// MISSING:
connectionId?: string; // Need for incoming connection
triggeredConnections?: string[]; // Need for outgoing
}
```
**Required Changes:**
1. Capture connectionId in TriggerChainRecorder
2. Capture triggered connections (next events in chain)
3. Store in TriggerEvent interface
### Implementation Plan
#### Phase 1: Data Capture (1 day)
**Update TriggerEvent interface:**
```typescript
interface TriggerEvent {
// ... existing fields
connectionId?: string; // ID of connection that triggered this
sourcePort?: string; // Output port name
targetPort?: string; // Input port name
nextEvents?: string[]; // IDs of events this triggered
}
```
**Update TriggerChainRecorder:**
- Capture connectionId from ViewerConnection data
- Parse source/target ports from connectionId
- Build forward links (nextEvents) during chain building
#### Phase 2: Connection Lookup (0.5 days)
**Add connection lookup to EventStep click handler:**
```typescript
// Find the connection from connectionId
const connection = component.graph.connections.find(/* match */);
if (connection) {
// Highlight it
}
```
**Challenge:** Connection ID format may not match graph model format
**Solution:** Use port + node matching as fallback
#### Phase 3: Highlighting System (1 day)
**Use existing HighlightManager:**
```typescript
// In EventStep.tsx click handler
HighlightManager.instance.highlightConnections(
[event.connectionId], // Incoming
{
channel: 'trigger-chain-incoming',
color: 'accent',
animated: true
}
);
HighlightManager.instance.highlightConnections(
event.nextConnectionIds, // Outgoing
{
channel: 'trigger-chain-outgoing',
color: 'success',
animated: false
}
);
```
**Fallback:** If HighlightManager doesn't support connections, extend it or use ViewerConnection highlighting
#### Phase 4: UI Controls (0.5 days)
**Add toggle in panel:**
- "Show connection flow" checkbox
- Enabled by default
- Persisted in user preferences
## Benefits for Step-by-Step Debugger
This enhancement directly enables step-by-step debugging by:
1. **Visual confirmation:** User sees exactly what will execute next
2. **Flow prediction:** Outgoing connections show multiple paths
3. **Debugging aid:** Easily spot unexpected connections
4. **Learning tool:** New users understand signal flow visually
## Edge Cases
1. **Cross-component signals:** Connection spans components
- Solution: Highlight in both components, navigate as needed
2. **Fan-out:** One signal triggers multiple nodes
- Solution: Highlight all outgoing connections
3. **Missing data:** ConnectionId not captured
- Solution: Graceful degradation (node-only highlighting)
4. **Performance:** Many connections highlighted
- Solution: Limit to 10 connections max, show "and X more"
## Success Criteria
- [ ] Clicking event highlights incoming connection
- [ ] Clicking event highlights all outgoing connections
- [ ] Highlighting persists until another event clicked
- [ ] Works across simple and complex signal chains
- [ ] Performance acceptable with 50+ nodes
- [ ] Graceful fallback if connection data missing
## Related Enhancements
- **ENHANCEMENT-step-by-step-debugger.md:** Uses this for visual feedback
- **KNOWN-ISSUES.md:** Connection data capture depends on event filtering fixes
## Files to Modify
- `packages/noodl-editor/src/editor/src/utils/triggerChain/types.ts`
- `packages/noodl-editor/src/editor/src/utils/triggerChain/TriggerChainRecorder.ts`
- `packages/noodl-editor/src/editor/src/views/panels/TriggerChainDebuggerPanel/components/EventStep.tsx`
- Potentially: `packages/noodl-editor/src/editor/src/services/HighlightManager.ts` (if connection support needed)

View File

@@ -0,0 +1,287 @@
# VIEW-003: Trigger Chain Debugger - Known Issues
## Status: ⚠️ UNSTABLE - REQUIRES INVESTIGATION
**Last Updated:** January 4, 2026
This document tracks critical bugs and issues discovered during testing that require investigation and fixing before VIEW-003 can be considered production-ready.
---
## 🔴 Critical Issues
### Issue #1: Deduplication Too Aggressive
**Status:** CRITICAL - Causing data loss
**Priority:** HIGH
**Discovered:** January 4, 2026
**Problem:**
The 5ms deduplication threshold implemented to fix "duplicate events" is now **dropping legitimate signal steps**. Real events that should be captured are being incorrectly filtered out.
**Symptoms:**
- Recording shows fewer events than actually occurred
- Missing steps in signal chains
- Incomplete trigger sequences in timeline
- User reports: "some actual signal steps missing from the recording"
**Root Cause (Hypothesis):**
- ViewerConnection's `connectiondebugpulse` handler may fire multiple times legitimately for:
- Rapid sequential signals (e.g., button click → show toast → navigate)
- Fan-out patterns (one signal triggering multiple downstream nodes)
- Component instantiation events
- Our deduplication logic can't distinguish between:
- **True duplicates:** Same event sent multiple times by ViewerConnection bug
- **Legitimate rapid events:** Multiple distinct events happening within 5ms
**Impact:**
- Feature is unreliable for debugging
- Cannot trust recorded data
- May miss critical steps in complex flows
**Required Investigation:**
1. Add verbose debug logging to `ViewerConnection.ts``connectiondebugpulse` handler
2. Capture ALL raw events before deduplication with timestamps
3. Analyze patterns across different scenarios:
- Simple button click → single action
- Button click → multiple chained actions
- Data flow through multiple nodes
- Component navigation events
- Hover/focus events (should these even be recorded?)
4. Determine if ViewerConnection bug exists vs legitimate high-frequency events
5. Design smarter deduplication strategy (see Investigation Plan below)
**Files Affected:**
- `packages/noodl-editor/src/editor/src/utils/triggerChain/TriggerChainRecorder.ts`
- `packages/noodl-editor/src/editor/src/ViewerConnection.ts`
---
### Issue #2: Event Filtering Strategy Undefined
**Status:** CRITICAL - No clear design
**Priority:** HIGH
**Discovered:** January 4, 2026
**Problem:**
There is **no defined strategy** for what types of events should be captured vs ignored. We're recording everything that comes through `connectiondebugpulse`, which may include:
- Visual updates (not relevant to signal flow)
- Hover/mouse events (noise)
- Render cycles (noise)
- Layout recalculations (noise)
- Legitimate signal triggers (SIGNAL - what we want!)
**Symptoms:**
- Event count explosion (40 events for simple actions)
- Timeline cluttered with irrelevant events
- Hard to find actual signal flow in the noise
- Performance concerns with recording high-frequency events
**Impact:**
- Feature unusable for debugging complex flows
- Cannot distinguish signal from noise
- Recording performance may degrade with complex projects
**Required Investigation:**
1. **Categorize all possible event types:**
- What does `connectiondebugpulse` actually send?
- What are the characteristics of each event type?
- Can we identify event types from connectionId format?
2. **Define filtering rules:**
- What makes an event a "signal trigger"?
- What events should be ignored?
- Should we have recording modes (all vs signals-only)?
3. **Test scenarios to document:**
- Button click → Show Toast
- REST API call → Update UI
- Navigation between pages
- Data binding updates
- Component lifecycle events
- Timer triggers
- User input (typing, dragging)
4. **Design decisions needed:**
- Should we filter at capture time or display time?
- Should we expose filter controls to user?
- Should we categorize events visually in timeline?
**Files Affected:**
- `packages/noodl-editor/src/editor/src/utils/triggerChain/TriggerChainRecorder.ts`
- `packages/noodl-editor/src/editor/src/ViewerConnection.ts`
- `packages/noodl-editor/src/editor/src/views/panels/TriggerChainDebuggerPanel/` (UI for filtering)
---
## 📋 Investigation Plan
### Phase 1: Data Collection (1-2 days)
**Goal:** Understand what we're actually receiving
1. **Add comprehensive debug logging:**
```typescript
// In ViewerConnection.ts
if (triggerChainRecorder.isRecording()) {
console.log('🔥 RAW EVENT:', {
connectionId,
timestamp: performance.now(),
extracted_uuids: uuids,
found_node: foundNode?.type?.name
});
content.connectionsToPulse.forEach((connectionId: string) => {
triggerChainRecorder.captureConnectionPulse(connectionId);
});
}
```
2. **Create test scenarios:**
- Simple: Button → Show Toast
- Medium: Button → REST API → Update Text
- Complex: Navigation → Load Data → Populate List
- Edge case: Rapid button clicks
- Edge case: Hover + Click interactions
3. **Capture and analyze:**
- Run each scenario
- Export console logs
- Count events by type
- Identify patterns in connectionId format
- Measure timing between events
### Phase 2: Pattern Analysis (1 day)
**Goal:** Categorize events and identify duplicates vs signals
1. **Categorize captured events:**
- Group by connectionId patterns
- Group by timing (< 1ms, 1-5ms, 5-50ms, > 50ms apart)
- Group by node type
- Group by component
2. **Identify true duplicates:**
- Events with identical connectionId and data
- Events within < 1ms (same frame)
- Determine if ViewerConnection bug exists
3. **Identify signal patterns:**
- What do button click signals look like?
- What do data flow signals look like?
- What do navigation signals look like?
4. **Identify noise patterns:**
- Render updates?
- Hover events?
- Focus events?
- Animation frame callbacks?
### Phase 3: Design Solution (1 day)
**Goal:** Design intelligent filtering strategy
1. **Deduplication Strategy:**
- Option A: Per-connectionId + timestamp threshold (current approach)
- Option B: Per-event-type + different thresholds
- Option C: Semantic deduplication (same source node + same data = duplicate)
- **Decision:** Choose based on Phase 1-2 findings
2. **Filtering Strategy:**
- Option A: Capture all, filter at display time (user control)
- Option B: Filter at capture time (performance)
- Option C: Hybrid (capture signals only, but allow "verbose mode")
- **Decision:** Choose based on performance measurements
3. **Event Classification:**
- Add `eventCategory` to TriggerEvent type
- Categories: `'signal' | 'data-flow' | 'visual' | 'lifecycle' | 'noise'`
- Visual indicators in timeline (colors, icons)
### Phase 4: Implementation (2-3 days)
1. Implement chosen deduplication strategy
2. Implement event filtering/classification
3. Add UI controls for filter toggles (if needed)
4. Update documentation
### Phase 5: Testing & Validation (1 day)
1. Test all scenarios from Phase 1
2. Verify event counts are accurate
3. Verify no legitimate signals are dropped
4. Verify duplicates are eliminated
5. Verify performance is acceptable
---
## 🎯 Success Criteria
Before marking VIEW-003 as stable:
- [ ] Can record button click → toast action with accurate event count (5-10 events max)
- [ ] No legitimate signal steps are dropped
- [ ] True duplicates are consistently filtered
- [ ] Event timeline is readable and useful
- [ ] Recording doesn't impact preview performance
- [ ] Deduplication strategy is documented and tested
- [ ] Event filtering rules are clear and documented
- [ ] User can distinguish signal flow from noise
---
## 📚 Related Documentation
- `CHANGELOG.md` - Implementation history
- `ENHANCEMENT-connection-highlighting.md` - Visual flow feature proposal
- `ENHANCEMENT-step-by-step-debugger.md` - Step-by-step execution proposal
- `dev-docs/reference/DEBUG-INFRASTRUCTURE.md` - ViewerConnection architecture
---
## 🔧 Current Workarounds
**For developers testing VIEW-003:**
1. **Expect inaccurate event counts** - Feature is unstable
2. **Cross-reference with manual testing** - Don't trust timeline alone
3. **Look for missing steps** - Some events may be dropped
4. **Avoid rapid interactions** - May trigger worst-case deduplication bugs
**For users:**
- Feature is marked experimental for a reason
- Use for general observation, not precise debugging
- Report anomalies to help with investigation
---
## 💡 Notes for Future Implementation
When fixing these issues, consider:
1. **Connection metadata:** Can we get more info from ViewerConnection about event type?
2. **Runtime instrumentation:** Should we add explicit "signal fired" events from runtime?
3. **Performance monitoring:** Add metrics for recording overhead
4. **User feedback:** Add UI indication when events are filtered
5. **Debug mode:** Add "raw event log" panel for investigation

View File

@@ -0,0 +1,212 @@
# VIEW-005: Data Lineage Panel - NOT PRODUCTION READY
**Status**: ⚠️ **NOT PRODUCTION READY** - Requires significant debugging and rework
**Date Marked**: January 4, 2026
---
## Summary
The Data Lineage panel was developed to trace data flow upstream (sources) and downstream (destinations) through the node graph. While the core engine and UI components were built, the feature has been **disabled from production** due to persistent issues that make it unusable in its current state.
---
## Issues Identified
### 1. **Event Handling / Timing Issues**
- Context menu event fires but panel shows "No node selected"
- Node selection state doesn't propagate correctly to the panel
- Attempted fixes with setTimeout and direct event passing didn't resolve the issue
### 2. **Excessive/Irrelevant Data in Results**
- Simple 3-node connections (Variable → String → Text) show 40+ upstream steps
- All unconnected ports being treated as "sources"
- Signal ports, metadata ports, and visual style properties included in traces
- Results are overwhelming and impossible to understand
### 3. **Filtering Inadequate**
- Port filtering (signals, metadata) only partially effective
- Primary port mapping not working as intended
- Depth limiting (MAX_DEPTH) not preventing noise
---
## What Was Implemented
**Core Engine** (`graphAnalysis/lineage.ts`)
- Upstream/downstream tracing logic
- Component boundary crossing
- Connection resolution
**UI Components**
- `DataLineagePanel.tsx` - Main panel component
- `LineagePath.tsx` - Path display component
- `PathSummary.tsx` - Summary statistics
- React hooks for lineage calculation
**Integration** (Now Disabled)
- ~~Context menu "Show Data Lineage" option~~ (commented out)
- Sidebar panel registration
- EventDispatcher integration
- Canvas highlighting for lineage paths
---
## Attempted Fixes
### Fix Attempt 1: Port Filtering
**Approach**: Filter out signal ports (`changed`, `fetched`) and metadata ports (`name`, `savedValue`)
**Result**: ❌ Reduced some noise but didn't solve the fundamental issue
### Fix Attempt 2: Skip Unconnected Ports
**Approach**: Don't treat every unconnected input as a "source"
**Result**: ❌ Should have helped but issue persists
### Fix Attempt 3: Primary Ports Only
**Approach**: Only trace main data port (`value` for Variables, `text` for Text nodes)
**Result**: ❌ Not effective, still too much data
### Fix Attempt 4: Depth Limiting
**Approach**: Reduced MAX_DEPTH from 50 to 5
**Result**: ❌ Didn't prevent the proliferation of paths
### Fix Attempt 5: Event Timing
**Approach**: setTimeout wrapper, then removed it
**Result**: ❌ Neither approach fixed selection state issue
---
## What Needs to be Done
### Critical Issues to Fix
1. **Debug Selection State**
- Why doesn't the node ID reach the panel?
- Is the event system working correctly?
- Add comprehensive logging to trace the full event flow
2. **Rethink Tracing Algorithm**
- Current approach of "trace all ports unless filtered" is fundamentally flawed
- Need a "trace only connected ports" approach from the ground up
- Should only follow actual wire connections, not enumerate ports
3. **Better Port Classification**
- Distinguish between:
- **Data ports** (value, text, items)
- **Style ports** (color, fontSize, padding)
- **Event ports** (onClick, onHover)
- **Metadata ports** (name, id)
- Only trace data ports by default
4. **Smarter Termination Conditions**
- Don't mark unconnected ports as sources/sinks
- Only mark actual source nodes (String, Number, etc.) as sources
- Properly detect end of lineage chains
### Recommended Approach
**Start Fresh with a Focused Scope:**
1. **Phase 1**: Get the basics working for a single use case
- Simple Variable → connection → Text node
- Should show exactly 2-3 steps
- Must display correctly in panel
2. **Phase 2**: Add one complexity at a time
- Expression nodes (transformation)
- Component boundaries
- Multi-hop paths
3. **Phase 3**: Handle edge cases
- Cycles
- Multiple sources/destinations
- Different node types
**Test-Driven Development:**
- Write tests FIRST for each scenario
- Verify traced paths match expectations
- Don't move on until tests pass
---
## Current Code State
The code is **present but disabled**:
### Disabled
- Context menu option (commented out in `nodegrapheditor.ts` lines ~2585-2600)
- Users cannot access the feature
### Still Present
- Panel component (`DataLineagePanel/`)
- Lineage engine (`utils/graphAnalysis/lineage.ts`)
- Sidebar registration (panel exists but hidden)
- All UI styling
**To Re-enable**: Uncomment the context menu section in `nodegrapheditor.ts` after issues are fixed.
---
## Files Involved
### Core Logic
- `packages/noodl-editor/src/editor/src/utils/graphAnalysis/lineage.ts` - Tracing engine
- `packages/noodl-editor/src/editor/src/utils/graphAnalysis/traversal.ts` - Port connections
- `packages/noodl-editor/src/editor/src/utils/graphAnalysis/crossComponent.ts` - Boundary crossing
### UI Components
- `packages/noodl-editor/src/editor/src/views/panels/DataLineagePanel/DataLineagePanel.tsx`
- `packages/noodl-editor/src/editor/src/views/panels/DataLineagePanel/components/LineagePath.tsx`
- `packages/noodl-editor/src/editor/src/views/panels/DataLineagePanel/components/PathSummary.tsx`
- `packages/noodl-editor/src/editor/src/views/panels/DataLineagePanel/hooks/useDataLineage.ts`
### Integration Points
- `packages/noodl-editor/src/editor/src/views/nodegrapheditor.ts` - Context menu (disabled)
- `packages/noodl-editor/src/editor/src/router.setup.ts` - Sidebar routing
- `packages/noodl-editor/src/editor/src/models/sidebar/sidebarmodel.ts` - Panel registration
---
## Conclusion
This feature **needs substantial rework** before it can be production-ready. The current implementation is not salvageable with small fixes - it requires a fundamental rethink of the tracing algorithm and careful test-driven development.
**Estimated Effort**: 2-3 days of focused work with proper debugging and testing
**Priority**: Low - This is a "nice to have" feature, not critical functionality
---
## Related Documentation
- See: `dev-docs/tasks/phase-4-canvas-visualisation-views/CLINE-INSTRUCTIONS.md`
- See: `dev-docs/reference/LEARNINGS.md` for general debugging patterns
---
**Last Updated**: January 4, 2026
**Marked Not Ready By**: Cline (AI Assistant)
**Approved By**: Richard Osborne

View File

@@ -1,5 +1,14 @@
# VIEW-005: Data Lineage View
> ⚠️ **STATUS: NOT PRODUCTION READY**
>
> This feature has been **disabled** due to persistent issues. The code exists but is commented out.
> See [NOT-PRODUCTION-READY.md](./NOT-PRODUCTION-READY.md) for details on issues and what needs fixing.
>
> **Estimated rework needed:** 2-3 days
---
**View Type:** 🎨 Canvas Overlay (enhances existing canvas with highlighting)
## Overview
@@ -16,6 +25,7 @@ A complete trace of where any value originates and where it flows to, crossing c
## The Problem
In a complex Noodl project:
- Data comes from parent components, API calls, user input... but where exactly?
- A value passes through 5 transformations before reaching its destination
- Component boundaries hide the full picture
@@ -28,6 +38,7 @@ The question: "I'm looking at this `userName` value in a Text node. Where does i
## The Solution
A visual lineage trace that:
1. Shows the complete upstream path (all the way to the source)
2. Shows the complete downstream path (all the way to final usage)
3. Crosses component boundaries transparently
@@ -185,9 +196,9 @@ interface LineageResult {
type: string;
componentName: string;
};
upstream: LineagePath;
downstream: LineagePath[]; // Can branch to multiple destinations
downstream: LineagePath[]; // Can branch to multiple destinations
}
interface LineagePath {
@@ -200,9 +211,9 @@ interface LineageStep {
component: ComponentModel;
port: string;
portType: 'input' | 'output';
transformation?: string; // Description of what happens (.name, Expression, etc.)
isSource?: boolean; // True if this is the ultimate origin
isSink?: boolean; // True if this is a final destination
transformation?: string; // Description of what happens (.name, Expression, etc.)
isSource?: boolean; // True if this is the ultimate origin
isSink?: boolean; // True if this is a final destination
}
interface ComponentCrossing {
@@ -210,7 +221,7 @@ interface ComponentCrossing {
to: ComponentModel;
viaPort: string;
direction: 'into' | 'outof';
stepIndex: number; // Where in the path this crossing occurs
stepIndex: number; // Where in the path this crossing occurs
}
```
@@ -221,16 +232,16 @@ function buildLineage(
project: ProjectModel,
component: ComponentModel,
nodeId: string,
port?: string // Optional: specific port to trace
port?: string // Optional: specific port to trace
): LineageResult {
const node = component.graph.findNodeWithId(nodeId);
// Trace upstream (find sources)
const upstream = traceUpstream(project, component, node, port);
// Trace downstream (find destinations)
const downstream = traceDownstream(project, component, node, port);
return {
selectedNode: {
id: node.id,
@@ -252,22 +263,20 @@ function traceUpstream(
): LineagePath {
const steps: LineageStep[] = [];
const crossings: ComponentCrossing[] = [];
// Prevent infinite loops
const nodeKey = `${component.fullName}:${node.id}`;
if (visited.has(nodeKey)) {
return { steps, crossings };
}
visited.add(nodeKey);
// Get input connections
const inputs = port
? getConnectionsToPort(component, node.id, port)
: getAllInputConnections(component, node.id);
const inputs = port ? getConnectionsToPort(component, node.id, port) : getAllInputConnections(component, node.id);
for (const connection of inputs) {
const sourceNode = component.graph.findNodeWithId(connection.fromId);
steps.push({
node: sourceNode,
component,
@@ -275,7 +284,7 @@ function traceUpstream(
portType: 'output',
transformation: describeTransformation(sourceNode, connection.fromProperty)
});
// Check if this is a Component Input (crosses boundary)
if (sourceNode.type.name === 'Component Inputs') {
const parentInfo = findParentConnection(project, component, connection.fromProperty);
@@ -287,7 +296,7 @@ function traceUpstream(
direction: 'into',
stepIndex: steps.length
});
// Continue tracing in parent component
const parentLineage = traceUpstream(
project,
@@ -309,20 +318,20 @@ function traceUpstream(
steps[steps.length - 1].isSource = true;
}
}
return { steps, crossings };
}
function isSourceNode(node: NodeGraphNode): boolean {
// These node types are considered "sources" - don't trace further
const sourceTypes = [
'REST', // API response is a source
'Variable', // Unless we want to trace where it was set
'REST', // API response is a source
'Variable', // Unless we want to trace where it was set
'Object',
'Page Inputs',
'Receive Event',
'Function', // Function output is a source
'String', // Literal values
'Function', // Function output is a source
'String', // Literal values
'Number',
'Boolean'
];
@@ -342,6 +351,7 @@ function isSourceNode(node: NodeGraphNode): boolean {
4. Detect source nodes (REST, Variable, etc.)
**Verification:**
- [ ] Can trace simple linear chains
- [ ] Handles multiple inputs
- [ ] Stops at source nodes
@@ -355,6 +365,7 @@ function isSourceNode(node: NodeGraphNode): boolean {
5. Track component crossings
**Verification:**
- [ ] Crosses into parent components
- [ ] Crosses into child components
- [ ] Crossings tracked correctly
@@ -366,6 +377,7 @@ function isSourceNode(node: NodeGraphNode): boolean {
3. Track all destination paths
**Verification:**
- [ ] Finds all destinations
- [ ] Handles branching
- [ ] Crosses component boundaries
@@ -379,6 +391,7 @@ function isSourceNode(node: NodeGraphNode): boolean {
5. Add path summary
**Verification:**
- [ ] Lineage renders correctly
- [ ] Component sections clear
- [ ] Crossings visually distinct
@@ -391,6 +404,7 @@ function isSourceNode(node: NodeGraphNode): boolean {
4. Handle edge cases (orphan nodes, cycles)
**Verification:**
- [ ] Navigation works
- [ ] Context menu works
- [ ] Edge cases handled gracefully
@@ -437,10 +451,10 @@ While Data Lineage is primarily a **static analysis** tool (showing the graph st
### Static vs Live Mode
| Mode | What it shows | Runtime needed? |
|------|---------------|-----------------|
| **Static** | The *path* data takes through the graph | No |
| **Live** | The *path* + *actual current values* at each step | Yes |
| Mode | What it shows | Runtime needed? |
| ---------- | ------------------------------------------------- | --------------- |
| **Static** | The _path_ data takes through the graph | No |
| **Live** | The _path_ + _actual current values_ at each step | Yes |
### Live Value Display
@@ -471,6 +485,7 @@ This answers "where does this come from?" AND "what's the actual value right now
### Integration with Existing Debug Infrastructure
The live values can come from the same system that powers:
- **DebugInspector hover values** - Already shows live values on connection hover
- **Pinned inspectors** - Already tracks values over time
@@ -494,6 +509,7 @@ function getLiveValueForNode(nodeId: string, port: string): unknown {
### Syncing with Canvas Highlighting
When the user hovers over a step in the lineage view:
- Highlight that node on the canvas (using existing highlighting)
- If the node is in a different component, show a "navigate" prompt
- Optionally flash the connection path on canvas
@@ -509,12 +525,12 @@ When the user hovers over a step in the lineage view:
## Risks & Mitigations
| Risk | Mitigation |
|------|------------|
| Deep component nesting | Limit depth, show "continue" option |
| Cycles in graph | Track visited nodes, break cycles |
| Many branches overwhelm UI | Collapse by default, expand on demand |
| Performance on complex graphs | Cache results, lazy expansion |
| Risk | Mitigation |
| ----------------------------- | ------------------------------------- |
| Deep component nesting | Limit depth, show "continue" option |
| Cycles in graph | Track visited nodes, break cycles |
| Many branches overwhelm UI | Collapse by default, expand on demand |
| Performance on complex graphs | Cache results, lazy expansion |
---