Forker/OpenNoodl
1
0
Fork 0
mirror of https://github.com/The-Low-Code-Foundation/OpenNoodl.git synced 2026-03-08 01:53:30 +01:00
Code Issues Actions Packages Projects Releases 1 Wiki Activity

docs: update progress for TASK-007A/B and CF11-004/005

Browse Source 
- Phase 11 PROGRESS.md: Mark CF11-004/005 complete, TASK-007B complete
- TASK-007 README: Update testing checklist with completion status
- Both docs now accurately reflect current implementation state
This commit is contained in:
Richard Osborne
2026-01-15 17:34:30 +01:00
parent 8e49cbedc9
commit dd73b1339b
2 changed files with 505 additions and 399 deletions
Download Patch File Download Diff File Expand all files Collapse all files

152
dev-docs/tasks/phase-11-cloud-functions/PROGRESS.md Normal file
View File

@@ -0,0 +1,152 @@
# Phase 11: Cloud Functions - Progress Tracker
**Phase Status:** 🔵 In Progress
**Last Updated:** 2026-01-15
---
## Prerequisites Status
| Dependency | Status | Notes |
| ------------------------------------ | ----------- | ----------------------------------------- |
| Phase 5 TASK-007A (LocalSQL Adapter) | ✅ Complete | Unblocks CF11-004/005 (Execution History) |
| Phase 5 TASK-007B (Backend Server) | ✅ Complete | REST API + IPC working! |
| Phase 5 TASK-007C (Workflow Runtime) | ⬜ Next | Required for trigger nodes |
> ✅ **Backend server running!** Tested and verified working with in-memory fallback.
---
## Series Progress
### Series 1: Advanced Workflow Nodes (0/3)
| Task | Name | Status | Assignee | Notes |
| -------- | -------------------- | -------------- | -------- | --------------- |
| CF11-001 | Logic Nodes | ⬜ Not Started | - | Needs TASK-007C |
| CF11-002 | Error Handling Nodes | ⬜ Not Started | - | Needs TASK-007C |
| CF11-003 | Wait/Delay Nodes | ⬜ Not Started | - | Needs TASK-007C |
### Series 2: Execution History (2/4) ⭐ PRIORITY
| Task | Name | Status | Assignee | Notes |
| -------- | ---------------------------- | -------------- | -------- | --------------------------- |
| CF11-004 | Execution Storage Schema | ✅ Complete | - | ExecutionStore implemented |
| CF11-005 | Execution Logger Integration | ✅ Complete | - | ExecutionLogger implemented |
| CF11-006 | Execution History Panel UI | ⬜ Not Started | - | **Ready to start!** |
| CF11-007 | Canvas Execution Overlay | ⬜ Not Started | - | After CF11-006 |
### Series 3: Cloud Deployment (0/4)
| Task | Name | Status | Assignee | Notes |
| -------- | --------------------------- | -------------- | -------- | ----- |
| CF11-008 | Docker Container Builder | ⬜ Not Started | - | |
| CF11-009 | Fly.io Deployment Provider | ⬜ Not Started | - | |
| CF11-010 | Railway Deployment Provider | ⬜ Not Started | - | |
| CF11-011 | Cloud Deploy Panel UI | ⬜ Not Started | - | |
### Series 4: Monitoring (0/3)
| Task | Name | Status | Assignee | Notes |
| -------- | ------------------------- | -------------- | -------- | ----- |
| CF11-012 | Metrics Collection System | ⬜ Not Started | - | |
| CF11-013 | Monitoring Dashboard UI | ⬜ Not Started | - | |
| CF11-014 | Alerting System | ⬜ Not Started | - | |
### Series 5: Python/AI Runtime (0/5)
| Task | Name | Status | Assignee | Notes |
| -------- | --------------------- | -------------- | -------- | ----- |
| CF11-015 | Python Runtime Bridge | ⬜ Not Started | - | |
| CF11-016 | Python Core Nodes | ⬜ Not Started | - | |
| CF11-017 | Claude/OpenAI Nodes | ⬜ Not Started | - | |
| CF11-018 | LangGraph Agent Node | ⬜ Not Started | - | |
| CF11-019 | Language Toggle UI | ⬜ Not Started | - | |
---
## Status Legend
| Symbol | Meaning |
| ------ | ----------- |
| ⬜ | Not Started |
| 🔵 | In Progress |
| 🟡 | Blocked |
| ✅ | Complete |
---
## Overall Progress
```
Series 1 (Nodes): [░░░░░░░░░░] 0% (needs TASK-007C)
Series 2 (History): [█████░░░░░] 50% ⭐ CF11-006 ready!
Series 3 (Deploy): [░░░░░░░░░░] 0%
Series 4 (Monitor): [░░░░░░░░░░] 0%
Series 5 (Python): [░░░░░░░░░░] 0%
─────────────────────────────────────
Total Phase 11: [██░░░░░░░░] 10%
```
---
## Recent Updates
| Date | Update |
| ---------- | ----------------------------------------------------------- |
| 2026-01-15 | ✅ **TASK-007B Complete** - Backend server running on 8580! |
| 2026-01-15 | ✅ **CF11-005 Complete** - ExecutionLogger implemented |
| 2026-01-15 | ✅ **CF11-004 Complete** - ExecutionStore implemented |
| 2026-01-15 | ✅ **TASK-007A Complete** - LocalSQL Adapter implemented |
| 2026-01-15 | Phase 11 restructured - removed overlap with Phase 5 |
---
## Git Branches
| Branch | Status | Commits | Notes |
| ------------------------------------ | -------- | ------- | ------------------------ |
| `feature/task-007a-localsql-adapter` | Complete | 1 | LocalSQL adapter |
| `feature/cf11-004-execution-storage` | Complete | 1 | ExecutionStore |
| `feature/cf11-005-execution-logger` | Complete | 1 | ExecutionLogger |
| `feature/task-007b-backend-server` | Complete | 4 | Backend + IPC + fallback |
---
## Testing Commands
```javascript
// In DevTools (Cmd+E) after npm run dev:
const { ipcRenderer } = require('electron');
const backend = await ipcRenderer.invoke('backend:create', 'Test');
await ipcRenderer.invoke('backend:start', backend.id);
fetch(`http://localhost:${backend.port}/health`)
.then((r) => r.json())
.then(console.log);
// → {status: 'ok', backend: 'Test', id: '...', port: 8580}
```
---
## Next Steps
1. **CF11-006: Execution History Panel UI** - Build the panel to display execution history
2. **TASK-007C: Workflow Runtime** - Complete the CloudRunner for function execution
3. **CF11-007: Canvas Overlay** - Visual execution status on canvas
---
## Blockers
| Blocker | Impact | Resolution |
| -------------- | ------ | ---------- |
| None currently | - | - |
---
## Notes
- Backend server uses in-memory mock when better-sqlite3 unavailable (Python 3.14 issue)
- Can add `sql.js` for persistent SQLite without native compilation
- Execution History (Series 2) is the highest priority
- External integrations deferred to future phase (see FUTURE-INTEGRATIONS.md)

330
dev-docs/tasks/phase-5-multi-target-deployment/01-byob-backend/TASK-007-integrated-backend/README.md
View File

@@ -16,6 +16,7 @@ Implement a zero-configuration local backend that runs alongside the Nodegex edi
### The Problem ### The Problem
New users consistently hit a wall when they need backend functionality: New users consistently hit a wall when they need backend functionality:
- They don't want to immediately learn Docker - They don't want to immediately learn Docker
- They don't want to pay for cloud backends before validating their idea - They don't want to pay for cloud backends before validating their idea
- The cognitive overhead of "choose and configure a backend" kills momentum - The cognitive overhead of "choose and configure a backend" kills momentum
@@ -23,6 +24,7 @@ New users consistently hit a wall when they need backend functionality:
### The Solution ### The Solution
An integrated, zero-config local backend that: An integrated, zero-config local backend that:
1. Starts automatically with the editor (optional) 1. Starts automatically with the editor (optional)
2. Uses the **same visual node paradigm** for backend workflows 2. Uses the **same visual node paradigm** for backend workflows
3. Provides instant full-stack development capability 3. Provides instant full-stack development capability
@@ -30,12 +32,12 @@ An integrated, zero-config local backend that:
### Why This Is a Game-Changer ### Why This Is a Game-Changer
| Current State | With Local Backend | | Current State | With Local Backend |
|---------------|-------------------| | ------------------------------------------- | ---------------------------------------- |
| "How do I add a database?" → Complex answer | "It's built in, just use Database nodes" | | "How do I add a database?" → Complex answer | "It's built in, just use Database nodes" |
| Requires external services for testing | 100% offline development | | Requires external services for testing | 100% offline development |
| Backend = different paradigm/tools | Backend = same Noodl visual nodes | | Backend = different paradigm/tools | Backend = same Noodl visual nodes |
| Prototype → Production = migration pain | Clear, assisted migration path | | Prototype → Production = migration pain | Clear, assisted migration path |
--- ---
@@ -99,10 +101,10 @@ An integrated, zero-config local backend that:
"name": "My Todo App", "name": "My Todo App",
"version": "2.0", "version": "2.0",
"backend": { "backend": {
"type": "local", // "local" | "parse" | "external" "type": "local", // "local" | "parse" | "external"
"id": "backend-uuid-1", // Reference to ~/.noodl/backends/{id} "id": "backend-uuid-1", // Reference to ~/.noodl/backends/{id}
"settings": { "settings": {
"autoStart": true, // Start with editor "autoStart": true, // Start with editor
"port": 8577 "port": 8577
} }
} }
@@ -181,6 +183,7 @@ Create the SQLite-based CloudStore adapter that implements the existing interfac
#### A.1: SQLite Integration (4 hours) #### A.1: SQLite Integration (4 hours)
**Files to create:** **Files to create:**
``` ```
packages/noodl-runtime/src/api/adapters/ packages/noodl-runtime/src/api/adapters/
├── index.ts # Adapter registry ├── index.ts # Adapter registry
@@ -196,9 +199,11 @@ packages/noodl-runtime/src/api/adapters/
```typescript ```typescript
// LocalSQLAdapter.ts // LocalSQLAdapter.ts
import Database from 'better-sqlite3';
import { CloudStoreAdapter } from '../cloudstore-adapter';
import { EventEmitter } from 'events'; import { EventEmitter } from 'events';
import Database from 'better-sqlite3';
import { CloudStoreAdapter } from '../cloudstore-adapter';
export class LocalSQLAdapter implements CloudStoreAdapter { export class LocalSQLAdapter implements CloudStoreAdapter {
private db: Database.Database; private db: Database.Database;
@@ -212,7 +217,7 @@ export class LocalSQLAdapter implements CloudStoreAdapter {
async query(options: QueryOptions): Promise<QueryResult> { async query(options: QueryOptions): Promise<QueryResult> {
const { sql, params } = QueryBuilder.buildSelect(options); const { sql, params } = QueryBuilder.buildSelect(options);
const rows = this.db.prepare(sql).all(...params); const rows = this.db.prepare(sql).all(...params);
return rows.map(row => this.rowToRecord(row, options.collection)); return rows.map((row) => this.rowToRecord(row, options.collection));
} }
async create(options: CreateOptions): Promise<Record> { async create(options: CreateOptions): Promise<Record> {
@@ -293,12 +298,7 @@ export class QueryBuilder {
return conditions.join(' AND '); return conditions.join(' AND ');
} }
private static translateOperator( private static translateOperator(field: string, op: string, value: any, params: any[]): string {
field: string,
op: string,
value: any,
params: any[]
): string {
const col = this.escapeColumn(field); const col = this.escapeColumn(field);
switch (op) { switch (op) {
@@ -375,7 +375,7 @@ export class SchemaManager {
'objectId TEXT PRIMARY KEY', 'objectId TEXT PRIMARY KEY',
'createdAt TEXT DEFAULT CURRENT_TIMESTAMP', 'createdAt TEXT DEFAULT CURRENT_TIMESTAMP',
'updatedAt TEXT DEFAULT CURRENT_TIMESTAMP', 'updatedAt TEXT DEFAULT CURRENT_TIMESTAMP',
...schema.columns.map(col => this.columnToSQL(col)) ...schema.columns.map((col) => this.columnToSQL(col))
]; ];
const sql = `CREATE TABLE IF NOT EXISTS ${schema.name} (${columns.join(', ')})`; const sql = `CREATE TABLE IF NOT EXISTS ${schema.name} (${columns.join(', ')})`;
@@ -430,17 +430,17 @@ export class SchemaManager {
} }
async exportSchema(): Promise<TableSchema[]> { async exportSchema(): Promise<TableSchema[]> {
const tables = this.db.prepare( const tables = this.db
"SELECT name FROM sqlite_master WHERE type='table' AND name NOT LIKE 'sqlite_%'" .prepare("SELECT name FROM sqlite_master WHERE type='table' AND name NOT LIKE 'sqlite_%'")
).all() as { name: string }[]; .all() as { name: string }[];
return tables.map(t => this.getTableSchema(t.name)); return tables.map((t) => this.getTableSchema(t.name));
} }
async generatePostgresSQL(): Promise<string> { async generatePostgresSQL(): Promise<string> {
// Export schema as Postgres-compatible SQL for migration // Export schema as Postgres-compatible SQL for migration
const schemas = await this.exportSchema(); const schemas = await this.exportSchema();
return schemas.map(s => this.tableToPostgresSQL(s)).join('\n\n'); return schemas.map((s) => this.tableToPostgresSQL(s)).join('\n\n');
} }
} }
``` ```
@@ -452,7 +452,9 @@ export class SchemaManager {
import { CloudStoreAdapter } from '../cloudstore-adapter'; import { CloudStoreAdapter } from '../cloudstore-adapter';
import { LocalSQLAdapter } from './local-sql/LocalSQLAdapter'; import { LocalSQLAdapter } from './local-sql/LocalSQLAdapter';
import { ParseAdapter } from './parse/ParseAdapter'; // Existing, refactored import { ParseAdapter } from './parse/ParseAdapter';
// Existing, refactored
export type AdapterType = 'local' | 'parse' | 'external'; export type AdapterType = 'local' | 'parse' | 'external';
@@ -526,11 +528,11 @@ Extend the existing Express server to handle local backend operations.
```typescript ```typescript
// packages/noodl-editor/src/main/src/local-backend/LocalBackendServer.ts // packages/noodl-editor/src/main/src/local-backend/LocalBackendServer.ts
import express, { Express, Request, Response } from 'express';
import http from 'http'; import http from 'http';
import express, { Express, Request, Response } from 'express';
import { WebSocketServer, WebSocket } from 'ws'; import { WebSocketServer, WebSocket } from 'ws';
import { LocalSQLAdapter } from '@noodl/runtime/src/api/adapters/local-sql/LocalSQLAdapter';
import { CloudRunner } from '@noodl/cloud-runtime'; import { CloudRunner } from '@noodl/cloud-runtime';
import { LocalSQLAdapter } from '@noodl/runtime/src/api/adapters/local-sql/LocalSQLAdapter';
export interface LocalBackendConfig { export interface LocalBackendConfig {
id: string; id: string;
@@ -736,10 +738,7 @@ export class LocalBackendServer {
const files = await fs.readdir(this.config.workflowsPath); const files = await fs.readdir(this.config.workflowsPath);
for (const file of files) { for (const file of files) {
if (file.endsWith('.workflow.json')) { if (file.endsWith('.workflow.json')) {
const content = await fs.readFile( const content = await fs.readFile(path.join(this.config.workflowsPath, file), 'utf-8');
path.join(this.config.workflowsPath, file),
'utf-8'
);
const workflow = JSON.parse(content); const workflow = JSON.parse(content);
await this.cloudRunner.load(workflow); await this.cloudRunner.load(workflow);
} }
@@ -756,10 +755,11 @@ export class LocalBackendServer {
```typescript ```typescript
// packages/noodl-editor/src/main/src/local-backend/BackendManager.ts // packages/noodl-editor/src/main/src/local-backend/BackendManager.ts
import { LocalBackendServer, LocalBackendConfig } from './LocalBackendServer';
import { ipcMain } from 'electron';
import * as fs from 'fs/promises'; import * as fs from 'fs/promises';
import * as path from 'path'; import * as path from 'path';
import { ipcMain } from 'electron';
import { LocalBackendServer, LocalBackendConfig } from './LocalBackendServer';
export interface BackendMetadata { export interface BackendMetadata {
id: string; id: string;
@@ -782,11 +782,7 @@ export class BackendManager {
} }
constructor() { constructor() {
this.backendsPath = path.join( this.backendsPath = path.join(process.env.HOME || process.env.USERPROFILE || '', '.noodl', 'backends');
process.env.HOME || process.env.USERPROFILE || '',
'.noodl',
'backends'
);
this.setupIPC(); this.setupIPC();
} }
@@ -798,9 +794,7 @@ export class BackendManager {
ipcMain.handle('backend:start', (_, id: string) => this.startBackend(id)); ipcMain.handle('backend:start', (_, id: string) => this.startBackend(id));
ipcMain.handle('backend:stop', (_, id: string) => this.stopBackend(id)); ipcMain.handle('backend:stop', (_, id: string) => this.stopBackend(id));
ipcMain.handle('backend:status', (_, id: string) => this.getStatus(id)); ipcMain.handle('backend:status', (_, id: string) => this.getStatus(id));
ipcMain.handle('backend:export-schema', (_, id: string, format: string) => ipcMain.handle('backend:export-schema', (_, id: string, format: string) => this.exportSchema(id, format));
this.exportSchema(id, format)
);
} }
async listBackends(): Promise<BackendMetadata[]> { async listBackends(): Promise<BackendMetadata[]> {
@@ -840,16 +834,10 @@ export class BackendManager {
projectIds: [] projectIds: []
}; };
await fs.writeFile( await fs.writeFile(path.join(backendPath, 'config.json'), JSON.stringify(metadata, null, 2));
path.join(backendPath, 'config.json'),
JSON.stringify(metadata, null, 2)
);
// Create empty schema // Create empty schema
await fs.writeFile( await fs.writeFile(path.join(backendPath, 'schema.json'), JSON.stringify({ tables: [] }, null, 2));
path.join(backendPath, 'schema.json'),
JSON.stringify({ tables: [] }, null, 2)
);
return metadata; return metadata;
} }
@@ -860,9 +848,7 @@ export class BackendManager {
} }
const backendPath = path.join(this.backendsPath, id); const backendPath = path.join(this.backendsPath, id);
const config = JSON.parse( const config = JSON.parse(await fs.readFile(path.join(backendPath, 'config.json'), 'utf-8'));
await fs.readFile(path.join(backendPath, 'config.json'), 'utf-8')
);
const server = new LocalBackendServer({ const server = new LocalBackendServer({
id, id,
@@ -921,7 +907,7 @@ export class BackendManager {
private async findAvailablePort(): Promise<number> { private async findAvailablePort(): Promise<number> {
// Start from 8577 and find next available // Start from 8577 and find next available
const existingBackends = await this.listBackends(); const existingBackends = await this.listBackends();
const usedPorts = new Set(existingBackends.map(b => b.port)); const usedPorts = new Set(existingBackends.map((b) => b.port));
let port = 8577; let port = 8577;
while (usedPorts.has(port)) { while (usedPorts.has(port)) {
@@ -1326,9 +1312,9 @@ export class WorkflowCompiler {
} }
// Get all cloud/local workflow components // Get all cloud/local workflow components
const workflowComponents = project.getComponents().filter(c => const workflowComponents = project
c.name.startsWith('/#__cloud__/') || c.name.startsWith('/#__local__/') .getComponents()
); .filter((c) => c.name.startsWith('/#__cloud__/') || c.name.startsWith('/#__local__/'));
if (workflowComponents.length === 0) { if (workflowComponents.length === 0) {
return; return;
@@ -1345,9 +1331,7 @@ export class WorkflowCompiler {
delete exported.metadata?.styles; delete exported.metadata?.styles;
delete exported.componentIndex; delete exported.componentIndex;
const workflowName = component.name const workflowName = component.name.replace('/#__cloud__/', '').replace('/#__local__/', '');
.replace('/#__cloud__/', '')
.replace('/#__local__/', '');
// Send to backend // Send to backend
await window.electronAPI.invoke('backend:update-workflow', { await window.electronAPI.invoke('backend:update-workflow', {
@@ -1373,9 +1357,10 @@ Add backend management to the project launcher.
// packages/noodl-editor/src/editor/src/views/Launcher/BackendManager/BackendList.tsx // packages/noodl-editor/src/editor/src/views/Launcher/BackendManager/BackendList.tsx
import React, { useState, useEffect } from 'react'; import React, { useState, useEffect } from 'react';
import { BackendCard } from './BackendCard'; import { BackendCard } from './BackendCard';
import { CreateBackendDialog } from './CreateBackendDialog';
import styles from './BackendList.module.scss'; import styles from './BackendList.module.scss';
import { CreateBackendDialog } from './CreateBackendDialog';
interface Backend { interface Backend {
id: string; id: string;
@@ -1442,9 +1427,7 @@ export function BackendList() {
<div className={styles.container}> <div className={styles.container}>
<div className={styles.header}> <div className={styles.header}>
<h2>Local Backends</h2> <h2>Local Backends</h2>
<button onClick={() => setShowCreate(true)}> <button onClick={() => setShowCreate(true)}>+ New Backend</button>
+ New Backend
</button>
</div> </div>
{loading ? ( {loading ? (
@@ -1456,7 +1439,7 @@ export function BackendList() {
</div> </div>
) : ( ) : (
<div className={styles.list}> <div className={styles.list}>
{backends.map(backend => ( {backends.map((backend) => (
<BackendCard <BackendCard
key={backend.id} key={backend.id}
backend={backend} backend={backend}
@@ -1468,12 +1451,7 @@ export function BackendList() {
</div> </div>
)} )}
{showCreate && ( {showCreate && <CreateBackendDialog onClose={() => setShowCreate(false)} onCreate={handleCreate} />}
<CreateBackendDialog
onClose={() => setShowCreate(false)}
onCreate={handleCreate}
/>
)}
</div> </div>
); );
} }
@@ -1485,6 +1463,7 @@ export function BackendList() {
// packages/noodl-editor/src/editor/src/views/Launcher/ProjectCard/BackendSelector.tsx // packages/noodl-editor/src/editor/src/views/Launcher/ProjectCard/BackendSelector.tsx
import React, { useState, useEffect } from 'react'; import React, { useState, useEffect } from 'react';
import styles from './BackendSelector.module.scss'; import styles from './BackendSelector.module.scss';
interface Props { interface Props {
@@ -1506,38 +1485,37 @@ export function BackendSelector({ projectId, currentBackendId, onSelect }: Props
setBackends(list); setBackends(list);
} }
const currentBackend = backends.find(b => b.id === currentBackendId); const currentBackend = backends.find((b) => b.id === currentBackendId);
return ( return (
<div className={styles.selector}> <div className={styles.selector}>
<button <button className={styles.trigger} onClick={() => setIsOpen(!isOpen)}>
className={styles.trigger}
onClick={() => setIsOpen(!isOpen)}
>
<span className={styles.icon}>⚡</span> <span className={styles.icon}>⚡</span>
{currentBackend ? ( {currentBackend ? <span>{currentBackend.name}</span> : <span className={styles.placeholder}>No backend</span>}
<span>{currentBackend.name}</span>
) : (
<span className={styles.placeholder}>No backend</span>
)}
</button> </button>
{isOpen && ( {isOpen && (
<div className={styles.dropdown}> <div className={styles.dropdown}>
<button <button
className={styles.option} className={styles.option}
onClick={() => { onSelect(null); setIsOpen(false); }} onClick={() => {
onSelect(null);
setIsOpen(false);
}}
> >
No Backend No Backend
</button> </button>
<div className={styles.divider} /> <div className={styles.divider} />
{backends.map(backend => ( {backends.map((backend) => (
<button <button
key={backend.id} key={backend.id}
className={styles.option} className={styles.option}
onClick={() => { onSelect(backend.id); setIsOpen(false); }} onClick={() => {
onSelect(backend.id);
setIsOpen(false);
}}
> >
{backend.name} {backend.name}
{backend.id === currentBackendId && ' ✓'} {backend.id === currentBackendId && ' ✓'}
@@ -1570,6 +1548,7 @@ export function BackendSelector({ projectId, currentBackendId, onSelect }: Props
// packages/noodl-editor/src/editor/src/views/BackendPanel/ExportWizard.tsx // packages/noodl-editor/src/editor/src/views/BackendPanel/ExportWizard.tsx
import React, { useState } from 'react'; import React, { useState } from 'react';
import styles from './ExportWizard.module.scss'; import styles from './ExportWizard.module.scss';
type ExportFormat = 'postgres' | 'supabase' | 'pocketbase' | 'json'; type ExportFormat = 'postgres' | 'supabase' | 'pocketbase' | 'json';
@@ -1589,19 +1568,11 @@ export function ExportWizard({ backendId, onClose }: Props) {
setLoading(true); setLoading(true);
try { try {
const schema = await window.electronAPI.invoke( const schema = await window.electronAPI.invoke('backend:export-schema', backendId, format);
'backend:export-schema',
backendId,
format
);
let data = ''; let data = '';
if (includeData) { if (includeData) {
data = await window.electronAPI.invoke( data = await window.electronAPI.invoke('backend:export-data', backendId, format === 'json' ? 'json' : 'sql');
'backend:export-data',
backendId,
format === 'json' ? 'json' : 'sql'
);
} }
setResult(schema + (data ? '\n\n-- DATA\n' + data : '')); setResult(schema + (data ? '\n\n-- DATA\n' + data : ''));
@@ -1633,7 +1604,7 @@ export function ExportWizard({ backendId, onClose }: Props) {
<div className={styles.section}> <div className={styles.section}>
<label>Export Format</label> <label>Export Format</label>
<select value={format} onChange={e => setFormat(e.target.value as ExportFormat)}> <select value={format} onChange={(e) => setFormat(e.target.value as ExportFormat)}>
<option value="postgres">PostgreSQL</option> <option value="postgres">PostgreSQL</option>
<option value="supabase">Supabase (with RLS)</option> <option value="supabase">Supabase (with RLS)</option>
<option value="pocketbase">PocketBase</option> <option value="pocketbase">PocketBase</option>
@@ -1643,21 +1614,13 @@ export function ExportWizard({ backendId, onClose }: Props) {
<div className={styles.section}> <div className={styles.section}>
<label> <label>
<input <input type="checkbox" checked={includeData} onChange={(e) => setIncludeData(e.target.checked)} />
type="checkbox"
checked={includeData}
onChange={e => setIncludeData(e.target.checked)}
/>
Include sample data (for testing) Include sample data (for testing)
</label> </label>
</div> </div>
{!result ? ( {!result ? (
<button <button className={styles.exportBtn} onClick={handleExport} disabled={loading}>
className={styles.exportBtn}
onClick={handleExport}
disabled={loading}
>
{loading ? 'Exporting...' : 'Generate Export'} {loading ? 'Exporting...' : 'Generate Export'}
</button> </button>
) : ( ) : (
@@ -1687,6 +1650,7 @@ export function ExportWizard({ backendId, onClose }: Props) {
// packages/noodl-editor/src/editor/src/views/Migration/ParseMigrationWizard.tsx // packages/noodl-editor/src/editor/src/views/Migration/ParseMigrationWizard.tsx
import React, { useState } from 'react'; import React, { useState } from 'react';
import styles from './ParseMigrationWizard.module.scss'; import styles from './ParseMigrationWizard.module.scss';
interface Props { interface Props {
@@ -1702,12 +1666,7 @@ interface Props {
type Step = 'confirm' | 'fetching' | 'review' | 'migrating' | 'complete'; type Step = 'confirm' | 'fetching' | 'review' | 'migrating' | 'complete';
export function ParseMigrationWizard({ export function ParseMigrationWizard({ projectId, parseConfig, onComplete, onCancel }: Props) {
projectId,
parseConfig,
onComplete,
onCancel
}: Props) {
const [step, setStep] = useState<Step>('confirm'); const [step, setStep] = useState<Step>('confirm');
const [schema, setSchema] = useState<any>(null); const [schema, setSchema] = useState<any>(null);
const [dataStats, setDataStats] = useState<any>(null); const [dataStats, setDataStats] = useState<any>(null);
@@ -1733,15 +1692,12 @@ export function ParseMigrationWizard({
// Get record counts // Get record counts
const stats: any = {}; const stats: any = {};
for (const cls of data.results) { for (const cls of data.results) {
const countRes = await fetch( const countRes = await fetch(`${parseConfig.endpoint}/classes/${cls.className}?count=1&limit=0`, {
`${parseConfig.endpoint}/classes/${cls.className}?count=1&limit=0`, headers: {
{ 'X-Parse-Application-Id': parseConfig.appId,
headers: { 'X-Parse-Master-Key': parseConfig.masterKey || ''
'X-Parse-Application-Id': parseConfig.appId,
'X-Parse-Master-Key': parseConfig.masterKey || ''
}
} }
); });
const countData = await countRes.json(); const countData = await countRes.json();
stats[cls.className] = countData.count; stats[cls.className] = countData.count;
} }
@@ -1759,9 +1715,7 @@ export function ParseMigrationWizard({
try { try {
// Create new local backend // Create new local backend
const backend = await window.electronAPI.invoke('backend:create', const backend = await window.electronAPI.invoke('backend:create', `Migrated from ${parseConfig.appId}`);
`Migrated from ${parseConfig.appId}`
);
setNewBackendId(backend.id); setNewBackendId(backend.id);
// Start it // Start it
@@ -1787,15 +1741,12 @@ export function ParseMigrationWizard({
const batchSize = 100; const batchSize = 100;
while (skip < count) { while (skip < count) {
const response = await fetch( const response = await fetch(`${parseConfig.endpoint}/classes/${className}?limit=${batchSize}&skip=${skip}`, {
`${parseConfig.endpoint}/classes/${className}?limit=${batchSize}&skip=${skip}`, headers: {
{ 'X-Parse-Application-Id': parseConfig.appId,
headers: { 'X-Parse-Master-Key': parseConfig.masterKey || ''
'X-Parse-Application-Id': parseConfig.appId,
'X-Parse-Master-Key': parseConfig.masterKey || ''
}
} }
); });
const data = await response.json(); const data = await response.json();
@@ -1819,11 +1770,7 @@ export function ParseMigrationWizard({
} }
// Render different steps... // Render different steps...
return ( return <div className={styles.wizard}>{/* Step UI here */}</div>;
<div className={styles.wizard}>
{/* Step UI here */}
</div>
);
} }
``` ```
@@ -1851,38 +1798,21 @@ export async function bundleBackend(options: BundleOptions): Promise<void> {
await fs.mkdir(path.join(outputPath, 'backend'), { recursive: true }); await fs.mkdir(path.join(outputPath, 'backend'), { recursive: true });
// Get backend config // Get backend config
const backendPath = path.join( const backendPath = path.join(process.env.HOME || '', '.noodl/backends', backendId);
process.env.HOME || '',
'.noodl/backends',
backendId
);
// Copy server code (pre-bundled) // Copy server code (pre-bundled)
const serverBundle = await getServerBundle(platform); const serverBundle = await getServerBundle(platform);
await fs.writeFile( await fs.writeFile(path.join(outputPath, 'backend', 'server.js'), serverBundle);
path.join(outputPath, 'backend', 'server.js'),
serverBundle
);
// Copy schema // Copy schema
await fs.copyFile( await fs.copyFile(path.join(backendPath, 'schema.json'), path.join(outputPath, 'backend', 'schema.json'));
path.join(backendPath, 'schema.json'),
path.join(outputPath, 'backend', 'schema.json')
);
// Copy workflows // Copy workflows
await fs.cp( await fs.cp(path.join(backendPath, 'workflows'), path.join(outputPath, 'backend', 'workflows'), { recursive: true });
path.join(backendPath, 'workflows'),
path.join(outputPath, 'backend', 'workflows'),
{ recursive: true }
);
// Optionally copy data // Optionally copy data
if (includeData) { if (includeData) {
await fs.copyFile( await fs.copyFile(path.join(backendPath, 'data', 'local.db'), path.join(outputPath, 'backend', 'data.db'));
path.join(backendPath, 'data', 'local.db'),
path.join(outputPath, 'backend', 'data.db')
);
} }
// Generate package.json // Generate package.json
@@ -1895,16 +1825,13 @@ export async function bundleBackend(options: BundleOptions): Promise<void> {
}, },
dependencies: { dependencies: {
'better-sqlite3': '^9.0.0', 'better-sqlite3': '^9.0.0',
'express': '^4.18.0', express: '^4.18.0',
'ws': '^8.0.0', ws: '^8.0.0',
'node-cron': '^3.0.0' 'node-cron': '^3.0.0'
} }
}; };
await fs.writeFile( await fs.writeFile(path.join(outputPath, 'backend', 'package.json'), JSON.stringify(packageJson, null, 2));
path.join(outputPath, 'backend', 'package.json'),
JSON.stringify(packageJson, null, 2)
);
// Generate startup script // Generate startup script
const startupScript = ` const startupScript = `
@@ -1926,22 +1853,13 @@ backend.stderr.on('data', (data) => console.error('[Backend]', data.toString()))
module.exports = { backend }; module.exports = { backend };
`; `;
await fs.writeFile( await fs.writeFile(path.join(outputPath, 'start-backend.js'), startupScript);
path.join(outputPath, 'start-backend.js'),
startupScript
);
} }
async function getServerBundle(platform: string): Promise<string> { async function getServerBundle(platform: string): Promise<string> {
// Return pre-compiled server bundle // Return pre-compiled server bundle
// This would be generated during editor build // This would be generated during editor build
const bundlePath = path.join( const bundlePath = path.join(__dirname, '..', 'resources', 'local-backend', `server.${platform}.bundle.js`);
__dirname,
'..',
'resources',
'local-backend',
`server.${platform}.bundle.js`
);
return fs.readFile(bundlePath, 'utf-8'); return fs.readFile(bundlePath, 'utf-8');
} }
``` ```
@@ -2080,34 +1998,45 @@ packages/noodl-viewer-cloud/src/nodes/index.ts
## Testing Checklist ## Testing Checklist
### LocalSQL Adapter ### LocalSQL Adapter ✅ COMPLETE (TASK-007A)
- [ ] Query with all operator types (equalTo, greaterThan, contains, etc.)
- [ ] Create/Save/Delete operations
- [ ] Relation operations
- [ ] Schema creation and migration
- [ ] Concurrent access handling
- [ ] Large dataset performance
### Local Backend Server - [x] Query with all operator types (equalTo, greaterThan, contains, etc.)
- [ ] REST endpoints respond correctly - [x] Create/Save/Delete operations
- [ ] WebSocket connections work - [x] Relation operations
- [x] Schema creation and migration
- [x] In-memory fallback when better-sqlite3 unavailable
- [ ] Concurrent access handling (needs real SQLite)
- [ ] Large dataset performance (needs real SQLite)
### Local Backend Server ✅ COMPLETE (TASK-007B)
- [x] REST endpoints respond correctly
- [x] Health check endpoint works
- [x] IPC handlers for create/start/stop/list
- [x] Server starts and responds on dynamic port
- [ ] WebSocket connections (basic structure, needs testing)
- [ ] Realtime events broadcast - [ ] Realtime events broadcast
- [ ] CloudRunner executes workflows - [ ] CloudRunner executes workflows (needs TASK-007C)
- [ ] Multiple backends can run simultaneously - [ ] Multiple backends can run simultaneously (needs testing)
### Editor Integration ### Editor Integration (Partial)
- [ ] Backend status shows in UI
- [ ] Start/Stop from launcher works - [x] Backend status available via IPC
- [x] Start/Stop from DevTools works
- [ ] Backend status shows in UI (needs UI)
- [ ] Start/Stop from launcher works (needs UI)
- [ ] Project-backend association persists - [ ] Project-backend association persists
- [ ] Workflow hot reload works - [ ] Workflow hot reload works (needs TASK-007C)
### Backward Compatibility ### Backward Compatibility
- [ ] Existing Parse projects load correctly - [ ] Existing Parse projects load correctly
- [ ] Parse adapter still functions - [ ] Parse adapter still functions
- [ ] Migration wizard works - [ ] Migration wizard works
- [ ] No regressions in existing functionality - [ ] No regressions in existing functionality
### Deployment ### Deployment
- [ ] Schema export to Postgres works - [ ] Schema export to Postgres works
- [ ] Schema export to Supabase works - [ ] Schema export to Supabase works
- [ ] Electron bundle includes backend - [ ] Electron bundle includes backend
@@ -2129,15 +2058,19 @@ packages/noodl-viewer-cloud/src/nodes/index.ts
## Risks & Mitigations ## Risks & Mitigations
### Risk: SQLite concurrency limitations ### Risk: SQLite concurrency limitations
**Mitigation**: Use WAL mode, implement connection pooling, document limitations **Mitigation**: Use WAL mode, implement connection pooling, document limitations
### Risk: Parse query syntax gaps ### Risk: Parse query syntax gaps
**Mitigation**: Comprehensive query translation layer with fallback warnings **Mitigation**: Comprehensive query translation layer with fallback warnings
### Risk: Workflow runtime differences ### Risk: Workflow runtime differences
**Mitigation**: Extensive testing, clear documentation of node compatibility **Mitigation**: Extensive testing, clear documentation of node compatibility
### Risk: Migration data loss ### Risk: Migration data loss
**Mitigation**: Backup prompts, rollback capability, staged migration **Mitigation**: Backup prompts, rollback capability, staged migration
--- ---
@@ -2147,8 +2080,29 @@ packages/noodl-viewer-cloud/src/nodes/index.ts
**Blocked by:** None (can start immediately) **Blocked by:** None (can start immediately)
**Blocks:** **Blocks:**
- Phase 5 external adapter implementations (Supabase, PocketBase) - Phase 5 external adapter implementations (Supabase, PocketBase)
- Future marketplace backend templates - Future marketplace backend templates
- **Phase 11: Cloud Functions & Workflow Automation**
### Phase 11 Dependency Note
Phase 11 (Cloud Functions) depends on TASK-007 sub-tasks A, B, and C:
| This Sub-task | Phase 11 Needs |
| ------------------------------ | ----------------------------------------------------- |
| **TASK-007A** (LocalSQL) | CF11-004 reuses SQLite patterns for execution history |
| **TASK-007B** (Backend Server) | Execution APIs will be added to this server |
| **TASK-007C** (CloudRunner) | All Phase 11 workflow nodes require this runtime |
| TASK-007D/E/F | Not blocking - can be done after Phase 11 starts |
**Recommended sequencing:**
1. Complete TASK-007A, 007B, 007C first (~45h)
2. Start Phase 11 Series 1 & 2 (Advanced Nodes + Execution History)
3. Return to TASK-007D/E/F later OR continue Phase 11
See: [Phase 11 README](../../../../phase-11-cloud-functions/README.md) for full details.
--- ---