OpenNoodl/dev-docs/tasks/phase-1-dependency-updates/TASK-002-legacy-project-migration/README.md

TASK-002: Legacy Project Migration & Backward Compatibility

Metadata

Field	Value
ID	TASK-002
Phase	Phase 1 - Foundation
Priority	🔴 Critical
Difficulty	🔴 Hard
Estimated Time	5-7 days
Prerequisites	TASK-001 (Dependency Updates)
Branch	task/002-legacy-project-migration

Objective

Develop a robust migration system that ensures all existing Noodl projects created with older versions of the editor (and older dependency versions) can be imported into the updated OpenNoodl editor without breaking changes or data loss.

Background

Why This Task Is Critical

Many Noodl users have production projects that they've built over months or years using previous versions of the Noodl editor. These projects may rely on:

• Older React version behavior (React 17 and earlier)
• Deprecated node APIs
• Legacy project file formats
• Older dependency APIs (e.g., react-instantsearch-hooks-web vs react-instantsearch)
• Previous runtime behaviors

When we update dependencies in TASK-001 (React 17 → 19, etc.), we risk breaking these existing projects. This is unacceptable for our user base. A user should be able to:

1. Install the new OpenNoodl editor
2. Open their 3-year-old Noodl project
3. Have it work exactly as before (or with minimal guided fixes)

The Stakes

• Users have business-critical applications built in Noodl
• Some users may have hundreds of hours invested in their projects
• Breaking backward compatibility could permanently lose users
• Our credibility as a fork depends on being a seamless upgrade path

How This Fits Into The Bigger Picture

This task ensures TASK-001 (dependency updates) doesn't create orphaned projects. It's a safety net that must be in place before we can confidently ship updated dependencies.

Current State

What We Know

• Projects are stored as JSON files (graph definitions, components, etc.)
• The runtime interprets these files at runtime
• Different Noodl versions may have different:
  • Node definitions
  • Property types
  • Connection formats
  • Metadata schemas

What We Don't Know Yet

• Exactly which project format versions exist in the wild
• How many breaking changes exist between versions
• Which node APIs have changed over time
• Whether there's existing version metadata in project files

Research Needed

• Analyze project file structure
• Document all project file schemas
• Compare old vs new node definitions
• Identify all breaking changes from dependency updates

Desired State

After this task is complete:

1. Seamless Import: Users can open any legacy Noodl project in the new editor
2. Auto-Migration: Projects are automatically upgraded to the new format when opened
3. CLI Tool: A command-line utility exists for batch migration and validation
4. No Breaking Changes: All existing node connections and logic work as before
5. Clear Warnings: If manual intervention is needed, users see clear guidance
6. Backup Safety: Original projects are backed up before migration
7. Validation: A test suite verifies migration works with sample projects

Scope

In Scope

• Document all Noodl project file formats
• Create a version detection system for projects
• Build a migration engine for auto-upgrading projects
• Develop a CLI tool for import/validation of legacy projects
• Create migration handlers for known breaking changes
• Build a validation test suite with sample projects
• Add user-facing warnings and guidance for edge cases
• Implement automatic backup before migration

Out of Scope

• Creating new node types (that's feature work)
• Fixing bugs in legacy projects (that's user responsibility)
• Supporting unofficial Noodl forks
• Migrating cloud/backend configurations (separate concern)

Technical Approach

Phase 1: Research & Analysis

Key Areas to Investigate

Area	Files to Examine	Goal
Project Structure	Sample .noodl project folders	Understand file organization
Graph Format	*.json graph files	Document schema
Node Definitions	packages/noodl-runtime/src/nodes/	Map all node types
Component Format	Component JSON files	Document structure
Metadata	Project metadata files	Find version indicators

Questions to Answer

1. Where is project version stored? (if at all)
2. What changed between Noodl releases?
3. Which nodes have breaking API changes?
4. What React 17 → 19 patterns affect project files?

Phase 2: Version Detection System

Create a system to identify what version of Noodl created a project:

interface ProjectVersion {
  editorVersion: string;      // e.g., "2.8.0"
  formatVersion: string;      // e.g., "1.2"
  runtimeVersion: string;     // e.g., "1.0.0"
  detectedFeatures: string[]; // Feature flags found
}

function detectProjectVersion(projectPath: string): ProjectVersion {
  // 1. Check explicit version metadata
  // 2. Infer from file structure
  // 3. Infer from node usage patterns
  // 4. Default to "unknown/legacy"
}

Phase 3: Migration Engine

Build a pluggable migration system:

interface Migration {
  id: string;
  fromVersion: string;
  toVersion: string;
  description: string;
  migrate: (project: ProjectData) => ProjectData;
  validate: (project: ProjectData) => ValidationResult;
}

class MigrationEngine {
  private migrations: Migration[] = [];
  
  registerMigration(migration: Migration): void;
  getMigrationPath(from: string, to: string): Migration[];
  migrateProject(project: ProjectData, targetVersion: string): MigrationResult;
}

Known Migrations Needed

From	To	Migration
React 17 patterns	React 19	Update any stored component patterns
Old node format	New node format	Transform node definitions
Legacy connections	New connections	Update connection schema

Phase 4: CLI Tool

Create a command-line tool for migration:

# Validate a project without modifying it
noodl-migrate validate ./my-project

# Migrate a project (creates backup first)
noodl-migrate upgrade ./my-project

# Migrate with specific target version
noodl-migrate upgrade ./my-project --to-version 3.0

# Batch migrate multiple projects
noodl-migrate batch-upgrade ./projects-folder

# Generate migration report
noodl-migrate report ./my-project > migration-report.md

CLI Implementation Location

packages/noodl-cli/
├── package.json
├── bin/
│   └── noodl-migrate.js
├── src/
│   ├── commands/
│   │   ├── validate.ts
│   │   ├── upgrade.ts
│   │   ├── batch-upgrade.ts
│   │   └── report.ts
│   ├── migrations/
│   │   ├── index.ts
│   │   ├── v17-to-v19-react.ts
│   │   ├── legacy-node-format.ts
│   │   └── ...
│   └── utils/
│       ├── backup.ts
│       ├── version-detect.ts
│       └── validation.ts
└── tests/
    └── ...

Phase 5: Editor Integration

Integrate migration into the editor's project opening flow:

// In project loading code
async function openProject(projectPath: string): Promise<Project> {
  const version = detectProjectVersion(projectPath);
  
  if (needsMigration(version)) {
    const result = await showMigrationDialog(projectPath, version);
    
    if (result === 'migrate') {
      await backupProject(projectPath);
      await migrateProject(projectPath);
    } else if (result === 'cancel') {
      return null;
    }
  }
  
  return loadProject(projectPath);
}

Key Files to Create

File	Purpose
packages/noodl-cli/	New package for CLI tool
packages/noodl-editor/src/editor/src/utils/migration/	Migration engine
packages/noodl-editor/src/editor/src/utils/migration/migrations/	Individual migrations
packages/noodl-editor/src/editor/src/views/MigrationDialog/	UI for migration prompts

Key Files to Modify

File	Changes
packages/noodl-editor/src/editor/src/models/projectmodel.js	Add migration check on load
Various node definitions	Document version requirements
package.json (root)	Add noodl-cli workspace

Implementation Steps

Step 1: Project Format Research (Day 1)

1. Collect sample projects from different Noodl versions
2. Document JSON schema for each file type
3. Identify version indicators in existing projects
4. Create comprehensive format documentation
5. Document in NOTES.md

Step 2: Breaking Changes Audit (Day 1-2)

1. List all dependency updates from TASK-001
2. For each update, identify breaking changes
3. Map breaking changes to project file impact
4. Create migration requirement list
5. Update README with findings

Step 3: Version Detection System (Day 2)

1. Create ProjectVersion type definitions
2. Implement version detection logic
3. Add fallback for unknown/legacy projects
4. Write unit tests for detection
5. Document in CHANGELOG.md

Step 4: Migration Engine Core (Day 3)

1. Design migration interface
2. Implement MigrationEngine class
3. Create migration registration system
4. Build migration path calculator
5. Add rollback capability
6. Write unit tests

Step 5: Individual Migrations (Day 3-4)

1. Create migration for React 17 → 19 patterns
2. Create migration for node format changes
3. Create migration for connection schema changes
4. Create migration for each identified breaking change
5. Write tests for each migration

Step 6: CLI Tool (Day 4-5)

1. Create noodl-cli package structure
2. Implement validate command
3. Implement upgrade command
4. Implement batch-upgrade command
5. Implement report command
6. Add backup functionality
7. Write CLI tests
8. Create user documentation

Step 7: Editor Integration (Day 5-6)

1. Create MigrationDialog component
2. Add migration check to project loading
3. Implement automatic backup
4. Add migration progress UI
5. Handle edge cases and errors
6. Manual testing

Step 8: Validation & Testing (Day 6-7)

1. Create test project corpus (various versions)
2. Run migration on all test projects
3. Verify migrated projects work correctly
4. Fix any discovered issues
5. Document edge cases

Testing Plan

Unit Tests

• Version detection correctly identifies project versions
• Migration engine calculates correct migration paths
• Each individual migration transforms data correctly
• Backup system creates valid copies
• Rollback restores original state

Integration Tests

• CLI tool works end-to-end
• Editor integration opens legacy projects
• Migration dialog flows work correctly
• Batch migration handles multiple projects

Manual Testing Scenarios

• Open a project from Noodl 2.0
• Open a project from Noodl 2.5
• Open a project from the last official release
• Open a project with complex node graphs
• Open a project with custom components
• Verify all nodes still work after migration
• Verify all connections still work
• Verify preview renders correctly
• Test CLI on real legacy projects

Test Project Corpus

Create or collect test projects representing:

• Minimal project (single page)
• Complex project (multiple pages, components)
• Project using deprecated nodes
• Project with custom JavaScript
• Project with cloud functions
• Project from each known Noodl version

Success Criteria

• Any legacy Noodl project can be opened in the new editor
• Migration happens automatically without data loss
• CLI tool successfully migrates 100% of test corpus
• Users receive clear guidance if manual action needed
• Original projects are backed up before modification
• All migrated projects pass validation
• No runtime errors in migrated projects
• Documentation explains the migration process

Risks & Mitigations

Risk	Impact	Probability	Mitigation
Unknown project formats exist	High	Medium	Comprehensive testing, graceful fallbacks
Some migrations are impossible	High	Low	Document limitations, provide manual guides
Performance issues with large projects	Medium	Medium	Streaming migration, progress indicators
Users don't understand prompts	Medium	Medium	Clear UX, detailed documentation
Edge cases cause data corruption	Critical	Low	Always backup first, validation checks
Can't find sample legacy projects	Medium	Medium	Reach out to community, create synthetic tests

Rollback Plan

If migration causes issues:

1. User-level: Restore from automatic backup
2. System-level: Revert migration code, keep projects in legacy mode
3. Feature flag: Add ability to disable auto-migration
4. Support path: Document manual migration steps for edge cases

Open Questions

1. Do we have access to legacy Noodl projects for testing?
2. Is there documentation of past Noodl version changes?
3. Should we support projects from unofficial Noodl forks?
4. What's the oldest Noodl version we need to support?
5. Should the CLI be a separate npm package or bundled?

---

Dependency Analysis Impact (from TASK-000)

Based on the TASK-000 Dependency Analysis, the following dependency changes have implications for legacy project migration:

Already Applied Changes (Need Testing)

Dependency	Change	Migration Impact
React 17 → 19	Breaking	Projects using React patterns may behave differently
react-instantsearch	Package renamed	Search-related custom components may need updates
Algoliasearch 4 → 5	API changes	Cloud functions using search may need migration

Future Changes (Plan Ahead)

These are NOT in TASK-001 but may require migration handling in the future:

Dependency	Potential Change	Migration Impact
Express 4 → 5	Breaking API	Backend/cloud functions using Express patterns
Electron 31 → 39	Native API changes	Desktop app behavior, IPC, file system access
Dugite 1 → 3	Git API overhaul	Version control operations, commit history
ESLint 8 → 9	Config format	Developer tooling (not runtime)

Migration Handlers to Consider

Based on the dependency analysis, consider creating migration handlers for:

1. React Concurrent Mode Patterns

   • Projects using legacy ReactDOM.render patterns
   • Timing-dependent component behaviors
   • Strict mode double-render issues

2. Search Service Integration

   • Projects using Algolia search
   • Custom search implementations
   • API response format expectations

3. Runtime Dependencies

   • Projects bundled with older noodl-runtime versions
   • Node definitions that expect old API patterns
   • Custom JavaScript nodes using deprecated patterns

Testing Considerations

When testing legacy project migration, specifically validate:

• React 19 concurrent rendering doesn't break existing animations
• useEffect cleanup timing changes don't cause memory leaks
• Search functionality works after react-instantsearch migration
• Custom nodes using old prop patterns still work
• Preview renders correctly in updated viewer

References

• TASK-000: Dependency Analysis (comprehensive dependency audit)
• TASK-001: Dependency Updates (lists breaking changes)
• TASK-000 Impact Matrix
• Noodl project file documentation (if exists)
• React 19 migration guide
• Community feedback on pain points

Notes for Future Developers

This task is foundational for OpenNoodl's success. Take the time to:

• Document everything you discover
• Be conservative with assumptions
• Test with real-world projects when possible
• Err on the side of not breaking things

If you're ever unsure whether a change might break legacy projects, don't make it without adding a migration path first.