mirror of
https://github.com/The-Low-Code-Foundation/OpenNoodl.git
synced 2026-01-12 23:32:55 +01:00
Started tasks to migrate runtime to React 19. Added phase 3 projects
This commit is contained in:
Richard Osborne
@@ -0,0 +1,481 @@
|
||||
# AI-001: AI Project Scaffolding
|
||||
|
||||
## Overview
|
||||
|
||||
Enable users to describe their project idea in natural language and have AI generate a complete project scaffold with pages, components, data models, and basic styling. This transforms the "blank canvas" experience into an intelligent starting point.
|
||||
|
||||
## Context
|
||||
|
||||
Currently, project creation offers:
|
||||
- Blank "Hello World" template
|
||||
- Pre-built template gallery (limited selection)
|
||||
- Manual component-by-component building
|
||||
|
||||
New users face a steep learning curve:
|
||||
- Don't know where to start
|
||||
- Overwhelmed by node options
|
||||
- No guidance on structure
|
||||
|
||||
AI scaffolding provides:
|
||||
- Describe idea → Get working structure
|
||||
- Industry best practices baked in
|
||||
- Learning through example
|
||||
- Faster time-to-prototype
|
||||
|
||||
### Existing Infrastructure
|
||||
|
||||
From `AiAssistantModel.ts`:
|
||||
```typescript
|
||||
// Existing AI templates
|
||||
docsTemplates = [
|
||||
{ label: 'REST API', template: 'rest' },
|
||||
{ label: 'Form Validation', template: 'function-form-validation' },
|
||||
{ label: 'AI Function', template: 'function' },
|
||||
// ...
|
||||
]
|
||||
```
|
||||
|
||||
From `TemplateRegistry`:
|
||||
```typescript
|
||||
// Download and extract project templates
|
||||
templateRegistry.download({ templateUrl }) → zipPath
|
||||
```
|
||||
|
||||
From `LocalProjectsModel`:
|
||||
```typescript
|
||||
// Create new project from template
|
||||
newProject(callback, { name, path, projectTemplate })
|
||||
```
|
||||
|
||||
## Requirements
|
||||
|
||||
### Functional Requirements
|
||||
|
||||
1. **Natural Language Input**
|
||||
- Free-form text description of project
|
||||
- Example prompts for inspiration
|
||||
- Clarifying questions from AI
|
||||
- Refinement through conversation
|
||||
|
||||
2. **Project Analysis**
|
||||
- Identify project type (app, dashboard, form, etc.)
|
||||
- Extract features and functionality
|
||||
- Determine data models needed
|
||||
- Suggest appropriate structure
|
||||
|
||||
3. **Scaffold Generation**
|
||||
- Create page structure
|
||||
- Generate component hierarchy
|
||||
- Set up navigation flow
|
||||
- Create placeholder data models
|
||||
- Apply appropriate styling
|
||||
|
||||
4. **Preview & Refinement**
|
||||
- Preview generated structure before creation
|
||||
- Modify/refine via chat
|
||||
- Accept or regenerate parts
|
||||
- Explain what was generated
|
||||
|
||||
5. **Project Creation**
|
||||
- Create actual Noodl project
|
||||
- Import generated components
|
||||
- Set up routing/navigation
|
||||
- Open in editor
|
||||
|
||||
### Non-Functional Requirements
|
||||
|
||||
- Generation completes in < 30 seconds
|
||||
- Works with Claude API (Anthropic)
|
||||
- Graceful handling of API errors
|
||||
- Clear progress indication
|
||||
- Cost-effective token usage
|
||||
|
||||
## Technical Approach
|
||||
|
||||
### 1. AI Scaffolding Service
|
||||
|
||||
```typescript
|
||||
// packages/noodl-editor/src/editor/src/services/AiScaffoldingService.ts
|
||||
|
||||
interface ProjectDescription {
|
||||
rawText: string;
|
||||
clarifications?: Record<string, string>;
|
||||
}
|
||||
|
||||
interface ScaffoldResult {
|
||||
projectType: ProjectType;
|
||||
pages: PageDefinition[];
|
||||
components: ComponentDefinition[];
|
||||
dataModels: DataModelDefinition[];
|
||||
navigation: NavigationDefinition;
|
||||
styling: StylingDefinition;
|
||||
explanation: string;
|
||||
}
|
||||
|
||||
interface PageDefinition {
|
||||
name: string;
|
||||
route: string;
|
||||
description: string;
|
||||
components: string[]; // Component names used
|
||||
layout: 'stack' | 'grid' | 'sidebar' | 'tabs';
|
||||
}
|
||||
|
||||
interface ComponentDefinition {
|
||||
name: string;
|
||||
type: 'visual' | 'logic' | 'data';
|
||||
description: string;
|
||||
inputs: PortDefinition[];
|
||||
outputs: PortDefinition[];
|
||||
children?: ComponentDefinition[];
|
||||
prefab?: string; // Use existing prefab if available
|
||||
}
|
||||
|
||||
class AiScaffoldingService {
|
||||
private static instance: AiScaffoldingService;
|
||||
|
||||
// Main flow
|
||||
async analyzeDescription(description: string): Promise<AnalysisResult>;
|
||||
async generateScaffold(description: ProjectDescription): Promise<ScaffoldResult>;
|
||||
async refineScaffold(scaffold: ScaffoldResult, feedback: string): Promise<ScaffoldResult>;
|
||||
|
||||
// Project creation
|
||||
async createProject(scaffold: ScaffoldResult, name: string, path: string): Promise<ProjectModel>;
|
||||
|
||||
// Conversation
|
||||
async askClarification(description: string): Promise<ClarificationQuestion[]>;
|
||||
async chat(messages: ChatMessage[]): Promise<ChatResponse>;
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Prompt Engineering
|
||||
|
||||
```typescript
|
||||
// packages/noodl-editor/src/editor/src/services/ai/prompts/scaffolding.ts
|
||||
|
||||
const SYSTEM_PROMPT = `You are an expert Noodl application architect.
|
||||
Your task is to analyze project descriptions and generate detailed scaffolds
|
||||
for visual low-code applications.
|
||||
|
||||
Noodl is a visual programming platform with:
|
||||
- Pages (screens/routes)
|
||||
- Components (reusable UI elements)
|
||||
- Nodes (visual programming blocks)
|
||||
- Data models (objects, arrays, variables)
|
||||
- Logic nodes (conditions, loops, functions)
|
||||
|
||||
When generating scaffolds, consider:
|
||||
1. User experience and navigation flow
|
||||
2. Data management and state
|
||||
3. Reusability of components
|
||||
4. Mobile-first responsive design
|
||||
5. Performance and loading states
|
||||
|
||||
Output JSON following the ScaffoldResult schema.`;
|
||||
|
||||
const ANALYSIS_PROMPT = `Analyze this project description and identify:
|
||||
1. Project type (app, dashboard, form, e-commerce, etc.)
|
||||
2. Main features/functionality
|
||||
3. User roles/personas
|
||||
4. Data entities needed
|
||||
5. Key user flows
|
||||
6. Potential complexity areas
|
||||
|
||||
Description: {description}`;
|
||||
|
||||
const SCAFFOLD_PROMPT = `Generate a complete Noodl project scaffold for:
|
||||
|
||||
Project Type: {projectType}
|
||||
Features: {features}
|
||||
Data Models: {dataModels}
|
||||
|
||||
Create:
|
||||
1. Page structure with routes
|
||||
2. Component hierarchy
|
||||
3. Navigation flow
|
||||
4. Data model definitions
|
||||
5. Styling theme
|
||||
|
||||
Use these available prefabs when appropriate:
|
||||
{availablePrefabs}`;
|
||||
```
|
||||
|
||||
### 3. Scaffold to Project Converter
|
||||
|
||||
```typescript
|
||||
// packages/noodl-editor/src/editor/src/services/ai/ScaffoldConverter.ts
|
||||
|
||||
class ScaffoldConverter {
|
||||
// Convert scaffold definitions to actual Noodl components
|
||||
async convertToProject(scaffold: ScaffoldResult): Promise<ProjectFiles> {
|
||||
const project = new ProjectModel();
|
||||
|
||||
// Create pages
|
||||
for (const page of scaffold.pages) {
|
||||
const pageComponent = await this.createPage(page);
|
||||
project.addComponent(pageComponent);
|
||||
}
|
||||
|
||||
// Create reusable components
|
||||
for (const component of scaffold.components) {
|
||||
const comp = await this.createComponent(component);
|
||||
project.addComponent(comp);
|
||||
}
|
||||
|
||||
// Set up navigation
|
||||
await this.setupNavigation(project, scaffold.navigation);
|
||||
|
||||
// Apply styling
|
||||
await this.applyStyles(project, scaffold.styling);
|
||||
|
||||
return project;
|
||||
}
|
||||
|
||||
private async createPage(page: PageDefinition): Promise<ComponentModel> {
|
||||
// Create component with page layout
|
||||
const component = ComponentModel.create({
|
||||
name: page.name,
|
||||
type: 'page'
|
||||
});
|
||||
|
||||
// Add layout container based on page.layout
|
||||
const layout = this.createLayout(page.layout);
|
||||
component.addChild(layout);
|
||||
|
||||
// Add referenced components
|
||||
for (const compName of page.components) {
|
||||
const ref = this.createComponentReference(compName);
|
||||
layout.addChild(ref);
|
||||
}
|
||||
|
||||
return component;
|
||||
}
|
||||
|
||||
private async createComponent(def: ComponentDefinition): Promise<ComponentModel> {
|
||||
// Check if we can use a prefab
|
||||
if (def.prefab) {
|
||||
return await this.importPrefab(def.prefab, def);
|
||||
}
|
||||
|
||||
// Create custom component
|
||||
const component = ComponentModel.create({
|
||||
name: def.name,
|
||||
type: def.type
|
||||
});
|
||||
|
||||
// Add ports
|
||||
for (const input of def.inputs) {
|
||||
component.addInput(input);
|
||||
}
|
||||
for (const output of def.outputs) {
|
||||
component.addOutput(output);
|
||||
}
|
||||
|
||||
// Add children
|
||||
if (def.children) {
|
||||
for (const child of def.children) {
|
||||
const childComp = await this.createComponent(child);
|
||||
component.addChild(childComp);
|
||||
}
|
||||
}
|
||||
|
||||
return component;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 4. UI Flow
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ Create New Project [×] │
|
||||
├─────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ○ Start from scratch │
|
||||
│ ○ Use a template │
|
||||
│ ● Describe your project (AI) │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Describe what you want to build... │ │
|
||||
│ │ │ │
|
||||
│ │ I want to build a task management app where users can create │ │
|
||||
│ │ projects, add tasks with due dates, and track progress with │ │
|
||||
│ │ a kanban board view. │ │
|
||||
│ │ │ │
|
||||
│ └─────────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ 💡 Examples: │
|
||||
│ • "A recipe app with categories and favorites" │
|
||||
│ • "An e-commerce dashboard with sales charts" │
|
||||
│ • "A booking system for a salon" │
|
||||
│ │
|
||||
│ [Cancel] [Generate Project] │
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### 5. Preview & Refinement
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ Project Preview [×] │
|
||||
├─────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ STRUCTURE │ CHAT │
|
||||
│ ┌─────────────────────────────────┐ │ ┌─────────────────────────────┐│
|
||||
│ │ 📁 Pages │ │ │ 🤖 I've created a task ││
|
||||
│ │ 📄 HomePage │ │ │ management app with: ││
|
||||
│ │ 📄 ProjectsPage │ │ │ ││
|
||||
│ │ 📄 KanbanBoard │ │ │ • 4 pages for navigation ││
|
||||
│ │ 📄 TaskDetail │ │ │ • Kanban board component ││
|
||||
│ │ │ │ │ • Task and Project models ││
|
||||
│ │ 📁 Components │ │ │ • Drag-and-drop ready ││
|
||||
│ │ 🧩 TaskCard │ │ │ ││
|
||||
│ │ 🧩 KanbanColumn │ │ │ Want me to add anything? ││
|
||||
│ │ 🧩 ProjectCard │ │ ├─────────────────────────────┤│
|
||||
│ │ 🧩 NavBar │ │ │ Add a calendar view too ││
|
||||
│ │ │ │ │ [Send]││
|
||||
│ │ 📁 Data Models │ │ └─────────────────────────────┘│
|
||||
│ │ 📊 Task │ │ │
|
||||
│ │ 📊 Project │ │ │
|
||||
│ │ 📊 User │ │ │
|
||||
│ └─────────────────────────────────┘ │ │
|
||||
│ │
|
||||
│ [Regenerate] [Edit Manually] [Create] │
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Files to Create
|
||||
|
||||
1. `packages/noodl-editor/src/editor/src/services/AiScaffoldingService.ts`
|
||||
2. `packages/noodl-editor/src/editor/src/services/ai/prompts/scaffolding.ts`
|
||||
3. `packages/noodl-editor/src/editor/src/services/ai/ScaffoldConverter.ts`
|
||||
4. `packages/noodl-editor/src/editor/src/services/ai/AnthropicClient.ts`
|
||||
5. `packages/noodl-core-ui/src/components/modals/AiProjectModal/AiProjectModal.tsx`
|
||||
6. `packages/noodl-core-ui/src/components/modals/AiProjectModal/ProjectDescriptionInput.tsx`
|
||||
7. `packages/noodl-core-ui/src/components/modals/AiProjectModal/ScaffoldPreview.tsx`
|
||||
8. `packages/noodl-core-ui/src/components/modals/AiProjectModal/RefinementChat.tsx`
|
||||
|
||||
## Files to Modify
|
||||
|
||||
1. `packages/noodl-editor/src/editor/src/views/projectsview.ts`
|
||||
- Add "Describe your project" option
|
||||
- Launch AiProjectModal
|
||||
|
||||
2. `packages/noodl-core-ui/src/preview/launcher/Launcher/Launcher.tsx`
|
||||
- Add AI project creation button
|
||||
|
||||
3. `packages/noodl-editor/src/editor/src/utils/LocalProjectsModel.ts`
|
||||
- Add `newProjectFromScaffold()` method
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Phase 1: AI Infrastructure
|
||||
1. Create AnthropicClient wrapper
|
||||
2. Implement prompt templates
|
||||
3. Set up API key management
|
||||
4. Create scaffolding service skeleton
|
||||
|
||||
### Phase 2: Scaffold Generation
|
||||
1. Implement analyzeDescription
|
||||
2. Implement generateScaffold
|
||||
3. Test with various descriptions
|
||||
4. Refine prompts based on results
|
||||
|
||||
### Phase 3: Scaffold Converter
|
||||
1. Implement page creation
|
||||
2. Implement component creation
|
||||
3. Implement navigation setup
|
||||
4. Implement styling application
|
||||
|
||||
### Phase 4: UI - Input Phase
|
||||
1. Create AiProjectModal
|
||||
2. Create ProjectDescriptionInput
|
||||
3. Add example prompts
|
||||
4. Integrate with launcher
|
||||
|
||||
### Phase 5: UI - Preview Phase
|
||||
1. Create ScaffoldPreview
|
||||
2. Create structure tree view
|
||||
3. Create RefinementChat
|
||||
4. Add edit/regenerate actions
|
||||
|
||||
### Phase 6: Project Creation
|
||||
1. Implement createProject
|
||||
2. Handle prefab imports
|
||||
3. Open project in editor
|
||||
4. Show success/onboarding
|
||||
|
||||
## Testing Checklist
|
||||
|
||||
- [ ] Description analysis extracts features correctly
|
||||
- [ ] Scaffold generation produces valid structure
|
||||
- [ ] Prefabs used when appropriate
|
||||
- [ ] Converter creates valid components
|
||||
- [ ] Pages have correct routing
|
||||
- [ ] Navigation works between pages
|
||||
- [ ] Styling applied consistently
|
||||
- [ ] Refinement chat updates scaffold
|
||||
- [ ] Project opens in editor
|
||||
- [ ] Error handling for API failures
|
||||
- [ ] Rate limiting handled gracefully
|
||||
|
||||
## Dependencies
|
||||
|
||||
- Anthropic API access (Claude)
|
||||
- COMP-002 (Built-in Prefabs) - for scaffold components
|
||||
|
||||
## Blocked By
|
||||
|
||||
- None (can start immediately)
|
||||
|
||||
## Blocks
|
||||
|
||||
- AI-002 (Component Suggestions)
|
||||
- AI-003 (Natural Language Editing)
|
||||
|
||||
## Estimated Effort
|
||||
|
||||
- AI infrastructure: 4-5 hours
|
||||
- Scaffold generation: 6-8 hours
|
||||
- Scaffold converter: 6-8 hours
|
||||
- UI input phase: 4-5 hours
|
||||
- UI preview phase: 5-6 hours
|
||||
- Project creation: 3-4 hours
|
||||
- Testing & refinement: 4-5 hours
|
||||
- **Total: 32-41 hours**
|
||||
|
||||
## Success Criteria
|
||||
|
||||
1. Users can describe project in natural language
|
||||
2. AI generates appropriate structure
|
||||
3. Preview shows clear scaffold
|
||||
4. Refinement chat enables adjustments
|
||||
5. Created project is functional
|
||||
6. Time from idea to working scaffold < 2 minutes
|
||||
|
||||
## Example Prompts & Outputs
|
||||
|
||||
### Example 1: Task Manager
|
||||
|
||||
**Input:** "A task management app where users can create projects, add tasks with due dates, and track progress with a kanban board"
|
||||
|
||||
**Output:**
|
||||
- Pages: Home, Projects, Kanban, TaskDetail
|
||||
- Components: TaskCard, KanbanColumn, ProjectCard, NavBar
|
||||
- Data: Task (title, description, dueDate, status), Project (name, tasks[])
|
||||
|
||||
### Example 2: Recipe App
|
||||
|
||||
**Input:** "A recipe app with categories, favorites, and a shopping list generator"
|
||||
|
||||
**Output:**
|
||||
- Pages: Home, Categories, RecipeDetail, Favorites, ShoppingList
|
||||
- Components: RecipeCard, CategoryTile, IngredientList, AddToFavoritesButton
|
||||
- Data: Recipe, Category, Ingredient, ShoppingItem
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
- Voice input for description
|
||||
- Screenshot/mockup to scaffold
|
||||
- Integration with design systems
|
||||
- Multi-language support
|
||||
- Template learning from user projects
|
||||
@@ -0,0 +1,507 @@
|
||||
# AI-002: AI Component Suggestions
|
||||
|
||||
## Overview
|
||||
|
||||
Provide intelligent, context-aware component suggestions as users build their projects. When a user is working on a component, AI analyzes the context and suggests relevant nodes, connections, or entire sub-components that would complement what they're building.
|
||||
|
||||
## Context
|
||||
|
||||
Currently, users must:
|
||||
- Know what node they need
|
||||
- Search through the node picker
|
||||
- Understand which nodes work together
|
||||
- Manually create common patterns
|
||||
|
||||
This creates friction for:
|
||||
- New users learning the platform
|
||||
- Experienced users building repetitive patterns
|
||||
- Anyone implementing common UI patterns
|
||||
|
||||
AI suggestions provide:
|
||||
- "What you might need next" recommendations
|
||||
- Common pattern recognition
|
||||
- Learning through suggestion
|
||||
- Faster workflow for experts
|
||||
|
||||
### Integration with Existing AI
|
||||
|
||||
From `AiAssistantModel.ts`:
|
||||
```typescript
|
||||
// Existing AI node templates
|
||||
templates: AiTemplate[] = docsTemplates.map(...)
|
||||
|
||||
// Activity tracking
|
||||
addActivity({ id, type, title, prompt, node, graph })
|
||||
```
|
||||
|
||||
This task extends the AI capabilities to work alongside normal editing, not just through dedicated AI nodes.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Functional Requirements
|
||||
|
||||
1. **Context Analysis**
|
||||
- Analyze current component structure
|
||||
- Identify incomplete patterns
|
||||
- Detect user intent from recent actions
|
||||
- Consider project-wide context
|
||||
|
||||
2. **Suggestion Types**
|
||||
- **Node suggestions**: "Add a Loading state?"
|
||||
- **Connection suggestions**: "Connect this to..."
|
||||
- **Pattern completion**: "Complete this form with validation?"
|
||||
- **Prefab suggestions**: "Use the Form Input prefab?"
|
||||
|
||||
3. **Suggestion Display**
|
||||
- Non-intrusive inline hints
|
||||
- Expandable detail panel
|
||||
- One-click insertion
|
||||
- Keyboard shortcuts
|
||||
|
||||
4. **Learning & Relevance**
|
||||
- Learn from user accepts/rejects
|
||||
- Improve relevance over time
|
||||
- Consider user skill level
|
||||
- Avoid repetitive suggestions
|
||||
|
||||
5. **Control & Settings**
|
||||
- Enable/disable suggestions
|
||||
- Suggestion frequency
|
||||
- Types of suggestions
|
||||
- Reset learned preferences
|
||||
|
||||
### Non-Functional Requirements
|
||||
|
||||
- Suggestions appear within 500ms
|
||||
- No blocking of user actions
|
||||
- Minimal API calls (batch/cache)
|
||||
- Works offline (basic patterns)
|
||||
|
||||
## Technical Approach
|
||||
|
||||
### 1. Suggestion Service
|
||||
|
||||
```typescript
|
||||
// packages/noodl-editor/src/editor/src/services/AiSuggestionService.ts
|
||||
|
||||
interface SuggestionContext {
|
||||
component: ComponentModel;
|
||||
selectedNodes: NodeGraphNode[];
|
||||
recentActions: EditorAction[];
|
||||
projectContext: ProjectContext;
|
||||
userPreferences: UserPreferences;
|
||||
}
|
||||
|
||||
interface Suggestion {
|
||||
id: string;
|
||||
type: 'node' | 'connection' | 'pattern' | 'prefab';
|
||||
confidence: number; // 0-1
|
||||
title: string;
|
||||
description: string;
|
||||
preview?: string; // Visual preview
|
||||
action: SuggestionAction;
|
||||
dismissable: boolean;
|
||||
}
|
||||
|
||||
interface SuggestionAction {
|
||||
type: 'insert_node' | 'create_connection' | 'insert_pattern' | 'import_prefab';
|
||||
payload: any;
|
||||
}
|
||||
|
||||
class AiSuggestionService {
|
||||
private static instance: AiSuggestionService;
|
||||
private suggestionCache: Map<string, Suggestion[]> = new Map();
|
||||
private userFeedback: UserFeedbackStore;
|
||||
|
||||
// Main API
|
||||
async getSuggestions(context: SuggestionContext): Promise<Suggestion[]>;
|
||||
async applySuggestion(suggestion: Suggestion): Promise<void>;
|
||||
async dismissSuggestion(suggestion: Suggestion): Promise<void>;
|
||||
|
||||
// Feedback
|
||||
recordAccept(suggestion: Suggestion): void;
|
||||
recordReject(suggestion: Suggestion): void;
|
||||
recordIgnore(suggestion: Suggestion): void;
|
||||
|
||||
// Settings
|
||||
setEnabled(enabled: boolean): void;
|
||||
setFrequency(frequency: SuggestionFrequency): void;
|
||||
getSuggestionSettings(): SuggestionSettings;
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Context Analyzer
|
||||
|
||||
```typescript
|
||||
// packages/noodl-editor/src/editor/src/services/ai/ContextAnalyzer.ts
|
||||
|
||||
interface AnalysisResult {
|
||||
componentType: ComponentType;
|
||||
currentPattern: Pattern | null;
|
||||
incompletePatterns: IncompletePattern[];
|
||||
missingConnections: MissingConnection[];
|
||||
suggestedEnhancements: Enhancement[];
|
||||
}
|
||||
|
||||
class ContextAnalyzer {
|
||||
// Pattern detection
|
||||
detectPatterns(component: ComponentModel): Pattern[];
|
||||
detectIncompletePatterns(component: ComponentModel): IncompletePattern[];
|
||||
|
||||
// Connection analysis
|
||||
findMissingConnections(nodes: NodeGraphNode[]): MissingConnection[];
|
||||
findOrphanedNodes(component: ComponentModel): NodeGraphNode[];
|
||||
|
||||
// Intent inference
|
||||
inferUserIntent(recentActions: EditorAction[]): UserIntent;
|
||||
|
||||
// Project context
|
||||
getRelatedComponents(component: ComponentModel): ComponentModel[];
|
||||
getDataModelContext(component: ComponentModel): DataModel[];
|
||||
}
|
||||
|
||||
// Common patterns to detect
|
||||
const PATTERNS = {
|
||||
FORM_INPUT: {
|
||||
nodes: ['TextInput', 'Label'],
|
||||
missing: ['Validation', 'ErrorDisplay'],
|
||||
suggestion: 'Add form validation?'
|
||||
},
|
||||
LIST_ITEM: {
|
||||
nodes: ['Repeater', 'Group'],
|
||||
missing: ['ItemClick', 'DeleteAction'],
|
||||
suggestion: 'Add item interactions?'
|
||||
},
|
||||
DATA_FETCH: {
|
||||
nodes: ['REST'],
|
||||
missing: ['LoadingState', 'ErrorState'],
|
||||
suggestion: 'Add loading and error states?'
|
||||
},
|
||||
// ... more patterns
|
||||
};
|
||||
```
|
||||
|
||||
### 3. Suggestion Engine
|
||||
|
||||
```typescript
|
||||
// packages/noodl-editor/src/editor/src/services/ai/SuggestionEngine.ts
|
||||
|
||||
class SuggestionEngine {
|
||||
private contextAnalyzer: ContextAnalyzer;
|
||||
private patternLibrary: PatternLibrary;
|
||||
private prefabMatcher: PrefabMatcher;
|
||||
|
||||
async generateSuggestions(context: SuggestionContext): Promise<Suggestion[]> {
|
||||
const suggestions: Suggestion[] = [];
|
||||
|
||||
// 1. Local pattern matching (no API)
|
||||
const localSuggestions = this.getLocalSuggestions(context);
|
||||
suggestions.push(...localSuggestions);
|
||||
|
||||
// 2. AI-powered suggestions (API call)
|
||||
if (this.shouldCallApi(context)) {
|
||||
const aiSuggestions = await this.getAiSuggestions(context);
|
||||
suggestions.push(...aiSuggestions);
|
||||
}
|
||||
|
||||
// 3. Prefab matching
|
||||
const prefabSuggestions = this.getPrefabSuggestions(context);
|
||||
suggestions.push(...prefabSuggestions);
|
||||
|
||||
// 4. Rank and filter
|
||||
return this.rankSuggestions(suggestions, context);
|
||||
}
|
||||
|
||||
private getLocalSuggestions(context: SuggestionContext): Suggestion[] {
|
||||
const analysis = this.contextAnalyzer.analyze(context.component);
|
||||
const suggestions: Suggestion[] = [];
|
||||
|
||||
// Pattern completion
|
||||
for (const incomplete of analysis.incompletePatterns) {
|
||||
suggestions.push({
|
||||
type: 'pattern',
|
||||
title: incomplete.completionTitle,
|
||||
description: incomplete.description,
|
||||
confidence: incomplete.confidence,
|
||||
action: {
|
||||
type: 'insert_pattern',
|
||||
payload: incomplete.completionNodes
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// Missing connections
|
||||
for (const missing of analysis.missingConnections) {
|
||||
suggestions.push({
|
||||
type: 'connection',
|
||||
title: `Connect ${missing.from} to ${missing.to}`,
|
||||
confidence: missing.confidence,
|
||||
action: {
|
||||
type: 'create_connection',
|
||||
payload: missing
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
return suggestions;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 4. UI Components
|
||||
|
||||
#### Inline Suggestion Hint
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ Canvas │
|
||||
│ │
|
||||
│ ┌─────────────┐ ┌─────────────┐ │
|
||||
│ │ TextInput │──────│ Variable │ │
|
||||
│ └─────────────┘ └─────────────┘ │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────────────────────────┐ │
|
||||
│ │ 💡 Add form validation? [+ Add]│ │
|
||||
│ │ Validate input and show errors │ │
|
||||
│ └─────────────────────────────────────────┘ │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
#### Suggestion Panel
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ 💡 Suggestions [×] │
|
||||
├─────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ Based on your current component: │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ 🧩 Complete Form Pattern [+ Apply] │ │
|
||||
│ │ Add validation, error states, and submit handling │ │
|
||||
│ │ Confidence: ████████░░ 85% │ │
|
||||
│ ├─────────────────────────────────────────────────────────────────┤ │
|
||||
│ │ 📦 Use "Form Input" Prefab [+ Apply] │ │
|
||||
│ │ Replace with pre-built form input component │ │
|
||||
│ │ Confidence: ███████░░░ 75% │ │
|
||||
│ ├─────────────────────────────────────────────────────────────────┤ │
|
||||
│ │ 🔗 Connect to Submit button [+ Apply] │ │
|
||||
│ │ Wire up the form submission flow │ │
|
||||
│ │ Confidence: ██████░░░░ 65% │ │
|
||||
│ └─────────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ [⚙️ Suggestion Settings] │
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### 5. Pattern Library
|
||||
|
||||
```typescript
|
||||
// packages/noodl-editor/src/editor/src/services/ai/PatternLibrary.ts
|
||||
|
||||
interface PatternDefinition {
|
||||
id: string;
|
||||
name: string;
|
||||
description: string;
|
||||
trigger: PatternTrigger;
|
||||
completion: PatternCompletion;
|
||||
examples: string[];
|
||||
}
|
||||
|
||||
const PATTERNS: PatternDefinition[] = [
|
||||
{
|
||||
id: 'form-validation',
|
||||
name: 'Form Validation',
|
||||
description: 'Add input validation with error display',
|
||||
trigger: {
|
||||
hasNodes: ['TextInput', 'Variable'],
|
||||
missingNodes: ['Function', 'Condition', 'Text'],
|
||||
nodeCount: { min: 2, max: 5 }
|
||||
},
|
||||
completion: {
|
||||
nodes: [
|
||||
{ type: 'Function', name: 'Validate' },
|
||||
{ type: 'Condition', name: 'IsValid' },
|
||||
{ type: 'Text', name: 'ErrorMessage' }
|
||||
],
|
||||
connections: [
|
||||
{ from: 'TextInput.value', to: 'Validate.input' },
|
||||
{ from: 'Validate.result', to: 'IsValid.condition' },
|
||||
{ from: 'IsValid.false', to: 'ErrorMessage.visible' }
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
id: 'loading-state',
|
||||
name: 'Loading State',
|
||||
description: 'Add loading indicator during async operations',
|
||||
trigger: {
|
||||
hasNodes: ['REST'],
|
||||
missingNodes: ['Condition', 'Group'],
|
||||
},
|
||||
completion: {
|
||||
nodes: [
|
||||
{ type: 'Variable', name: 'IsLoading' },
|
||||
{ type: 'Group', name: 'LoadingSpinner' },
|
||||
{ type: 'Condition', name: 'ShowContent' }
|
||||
],
|
||||
connections: [
|
||||
{ from: 'REST.fetch', to: 'IsLoading.set(true)' },
|
||||
{ from: 'REST.success', to: 'IsLoading.set(false)' },
|
||||
{ from: 'IsLoading.value', to: 'LoadingSpinner.visible' }
|
||||
]
|
||||
}
|
||||
},
|
||||
// ... more patterns
|
||||
];
|
||||
```
|
||||
|
||||
## Files to Create
|
||||
|
||||
1. `packages/noodl-editor/src/editor/src/services/AiSuggestionService.ts`
|
||||
2. `packages/noodl-editor/src/editor/src/services/ai/ContextAnalyzer.ts`
|
||||
3. `packages/noodl-editor/src/editor/src/services/ai/SuggestionEngine.ts`
|
||||
4. `packages/noodl-editor/src/editor/src/services/ai/PatternLibrary.ts`
|
||||
5. `packages/noodl-editor/src/editor/src/services/ai/PrefabMatcher.ts`
|
||||
6. `packages/noodl-core-ui/src/components/ai/SuggestionHint/SuggestionHint.tsx`
|
||||
7. `packages/noodl-core-ui/src/components/ai/SuggestionPanel/SuggestionPanel.tsx`
|
||||
8. `packages/noodl-core-ui/src/components/ai/SuggestionCard/SuggestionCard.tsx`
|
||||
|
||||
## Files to Modify
|
||||
|
||||
1. `packages/noodl-editor/src/editor/src/views/nodegrapheditor.js`
|
||||
- Hook into node selection/creation
|
||||
- Trigger suggestion generation
|
||||
- Display suggestion hints
|
||||
|
||||
2. `packages/noodl-editor/src/editor/src/pages/EditorPage/EditorPage.tsx`
|
||||
- Add suggestion panel toggle
|
||||
- Handle suggestion keybindings
|
||||
|
||||
3. `packages/noodl-editor/src/editor/src/models/AiAssistant/AiAssistantModel.ts`
|
||||
- Integrate suggestion service
|
||||
- Share context with AI nodes
|
||||
|
||||
4. `packages/noodl-editor/src/editor/src/stores/EditorSettings.ts`
|
||||
- Add suggestion settings
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Phase 1: Context Analysis
|
||||
1. Create ContextAnalyzer
|
||||
2. Implement pattern detection
|
||||
3. Implement connection analysis
|
||||
4. Test with various components
|
||||
|
||||
### Phase 2: Pattern Library
|
||||
1. Define pattern schema
|
||||
2. Create initial patterns (10-15)
|
||||
3. Implement pattern matching
|
||||
4. Test pattern triggers
|
||||
|
||||
### Phase 3: Suggestion Engine
|
||||
1. Create SuggestionEngine
|
||||
2. Implement local suggestions
|
||||
3. Implement AI suggestions
|
||||
4. Add ranking/filtering
|
||||
|
||||
### Phase 4: UI - Inline Hints
|
||||
1. Create SuggestionHint component
|
||||
2. Position near relevant nodes
|
||||
3. Add apply/dismiss actions
|
||||
4. Animate appearance
|
||||
|
||||
### Phase 5: UI - Panel
|
||||
1. Create SuggestionPanel
|
||||
2. Create SuggestionCard
|
||||
3. Add settings access
|
||||
4. Handle keyboard shortcuts
|
||||
|
||||
### Phase 6: Feedback & Learning
|
||||
1. Track accept/reject
|
||||
2. Adjust confidence scores
|
||||
3. Improve relevance
|
||||
4. Add user settings
|
||||
|
||||
## Testing Checklist
|
||||
|
||||
- [ ] Patterns detected correctly
|
||||
- [ ] Suggestions appear at right time
|
||||
- [ ] Apply action works correctly
|
||||
- [ ] Dismiss removes suggestion
|
||||
- [ ] Inline hint positions correctly
|
||||
- [ ] Panel shows all suggestions
|
||||
- [ ] Settings persist
|
||||
- [ ] Works offline (local patterns)
|
||||
- [ ] API suggestions enhance local
|
||||
- [ ] Feedback recorded
|
||||
- [ ] Performance < 500ms
|
||||
|
||||
## Dependencies
|
||||
|
||||
- AI-001 (AI Project Scaffolding) - for AI infrastructure
|
||||
- COMP-002 (Built-in Prefabs) - for prefab matching
|
||||
|
||||
## Blocked By
|
||||
|
||||
- AI-001 (for AnthropicClient)
|
||||
|
||||
## Blocks
|
||||
|
||||
- AI-003 (Natural Language Editing)
|
||||
|
||||
## Estimated Effort
|
||||
|
||||
- Context analyzer: 4-5 hours
|
||||
- Pattern library: 4-5 hours
|
||||
- Suggestion engine: 5-6 hours
|
||||
- UI inline hints: 3-4 hours
|
||||
- UI panel: 4-5 hours
|
||||
- Feedback system: 3-4 hours
|
||||
- Testing & refinement: 4-5 hours
|
||||
- **Total: 27-34 hours**
|
||||
|
||||
## Success Criteria
|
||||
|
||||
1. Suggestions appear contextually
|
||||
2. Pattern completion works smoothly
|
||||
3. Prefab matching finds relevant prefabs
|
||||
4. Apply action inserts correctly
|
||||
5. Users can control suggestions
|
||||
6. Suggestions improve over time
|
||||
|
||||
## Pattern Categories
|
||||
|
||||
### Forms
|
||||
- Form validation
|
||||
- Form submission
|
||||
- Input formatting
|
||||
- Error display
|
||||
|
||||
### Data
|
||||
- Loading states
|
||||
- Error handling
|
||||
- Refresh/retry
|
||||
- Pagination
|
||||
|
||||
### Navigation
|
||||
- Page transitions
|
||||
- Breadcrumbs
|
||||
- Tab navigation
|
||||
- Modal flows
|
||||
|
||||
### Lists
|
||||
- Item selection
|
||||
- Delete/edit actions
|
||||
- Drag and drop
|
||||
- Filtering
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
- Real-time suggestions while typing
|
||||
- Team-shared patterns
|
||||
- Auto-apply for obvious patterns
|
||||
- Pattern creation from selection
|
||||
- AI-powered custom patterns
|
||||
@@ -0,0 +1,565 @@
|
||||
# AI-003: Natural Language Editing
|
||||
|
||||
## Overview
|
||||
|
||||
Enable users to modify their projects using natural language commands. Instead of manually finding and configuring nodes, users can say "make this button blue" or "add a loading spinner when fetching data" and have AI make the changes.
|
||||
|
||||
## Context
|
||||
|
||||
Current editing workflow:
|
||||
1. Select node in canvas
|
||||
2. Find property in sidebar
|
||||
3. Understand property options
|
||||
4. Make change
|
||||
5. Repeat for related nodes
|
||||
|
||||
Natural language editing:
|
||||
1. Select component or node
|
||||
2. Describe what you want
|
||||
3. AI makes the changes
|
||||
4. Review and accept/modify
|
||||
|
||||
This is especially powerful for:
|
||||
- Styling changes across multiple elements
|
||||
- Logic modifications that span nodes
|
||||
- Refactoring component structure
|
||||
- Complex multi-step changes
|
||||
|
||||
### Existing AI Foundation
|
||||
|
||||
From `AiAssistantModel.ts`:
|
||||
```typescript
|
||||
// Chat history for AI interactions
|
||||
class ChatHistory {
|
||||
messages: ChatMessage[];
|
||||
add(message: ChatMessage): void;
|
||||
}
|
||||
|
||||
// AI context per node
|
||||
class AiCopilotContext {
|
||||
template: AiTemplate;
|
||||
chatHistory: ChatHistory;
|
||||
node: NodeGraphNode;
|
||||
}
|
||||
```
|
||||
|
||||
## Requirements
|
||||
|
||||
### Functional Requirements
|
||||
|
||||
1. **Command Input**
|
||||
- Command palette (Cmd+K style)
|
||||
- Inline text input on selection
|
||||
- Voice input (optional)
|
||||
- Recent commands history
|
||||
|
||||
2. **Command Understanding**
|
||||
- Style changes: "make it red", "add shadow"
|
||||
- Structure changes: "add a header", "wrap in a card"
|
||||
- Logic changes: "show loading while fetching"
|
||||
- Data changes: "sort by date", "filter active items"
|
||||
|
||||
3. **Change Preview**
|
||||
- Show what will change before applying
|
||||
- Highlight affected nodes
|
||||
- Before/after comparison
|
||||
- Explanation of changes
|
||||
|
||||
4. **Change Application**
|
||||
- Apply changes atomically
|
||||
- Support undo/redo
|
||||
- Handle errors gracefully
|
||||
- Learn from corrections
|
||||
|
||||
5. **Scope Selection**
|
||||
- Selected node(s) only
|
||||
- Current component
|
||||
- Related components
|
||||
- Entire project
|
||||
|
||||
### Non-Functional Requirements
|
||||
|
||||
- Response time < 3 seconds
|
||||
- Changes are reversible
|
||||
- Works on any component type
|
||||
- Graceful degradation without API
|
||||
|
||||
## Technical Approach
|
||||
|
||||
### 1. Natural Language Command Service
|
||||
|
||||
```typescript
|
||||
// packages/noodl-editor/src/editor/src/services/NaturalLanguageService.ts
|
||||
|
||||
interface CommandContext {
|
||||
selection: NodeGraphNode[];
|
||||
component: ComponentModel;
|
||||
project: ProjectModel;
|
||||
recentCommands: Command[];
|
||||
}
|
||||
|
||||
interface Command {
|
||||
id: string;
|
||||
text: string;
|
||||
timestamp: Date;
|
||||
result: CommandResult;
|
||||
}
|
||||
|
||||
interface CommandResult {
|
||||
success: boolean;
|
||||
changes: Change[];
|
||||
explanation: string;
|
||||
undoAction?: () => void;
|
||||
}
|
||||
|
||||
interface Change {
|
||||
type: 'property' | 'node' | 'connection' | 'structure';
|
||||
target: string;
|
||||
before: any;
|
||||
after: any;
|
||||
description: string;
|
||||
}
|
||||
|
||||
class NaturalLanguageService {
|
||||
private static instance: NaturalLanguageService;
|
||||
|
||||
// Main API
|
||||
async parseCommand(text: string, context: CommandContext): Promise<ParsedCommand>;
|
||||
async previewChanges(command: ParsedCommand): Promise<ChangePreview>;
|
||||
async applyChanges(preview: ChangePreview): Promise<CommandResult>;
|
||||
async undoCommand(commandId: string): Promise<void>;
|
||||
|
||||
// Command history
|
||||
getRecentCommands(): Command[];
|
||||
searchCommands(query: string): Command[];
|
||||
|
||||
// Learning
|
||||
recordCorrection(commandId: string, correction: Change[]): void;
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Command Parser
|
||||
|
||||
```typescript
|
||||
// packages/noodl-editor/src/editor/src/services/ai/CommandParser.ts
|
||||
|
||||
interface ParsedCommand {
|
||||
intent: CommandIntent;
|
||||
targets: CommandTarget[];
|
||||
modifications: Modification[];
|
||||
confidence: number;
|
||||
}
|
||||
|
||||
enum CommandIntent {
|
||||
STYLE_CHANGE = 'style_change',
|
||||
STRUCTURE_CHANGE = 'structure_change',
|
||||
LOGIC_CHANGE = 'logic_change',
|
||||
DATA_CHANGE = 'data_change',
|
||||
CREATE = 'create',
|
||||
DELETE = 'delete',
|
||||
CONNECT = 'connect',
|
||||
UNKNOWN = 'unknown'
|
||||
}
|
||||
|
||||
interface CommandTarget {
|
||||
type: 'node' | 'component' | 'property' | 'connection';
|
||||
selector: string; // How to find it
|
||||
resolved?: any; // Actual reference
|
||||
}
|
||||
|
||||
class CommandParser {
|
||||
private patterns: CommandPattern[];
|
||||
|
||||
async parse(text: string, context: CommandContext): Promise<ParsedCommand> {
|
||||
// 1. Try local pattern matching first (fast)
|
||||
const localMatch = this.matchLocalPatterns(text);
|
||||
if (localMatch.confidence > 0.9) {
|
||||
return localMatch;
|
||||
}
|
||||
|
||||
// 2. Use AI for complex commands
|
||||
const aiParsed = await this.aiParse(text, context);
|
||||
|
||||
// 3. Merge and validate
|
||||
return this.mergeAndValidate(localMatch, aiParsed, context);
|
||||
}
|
||||
|
||||
private matchLocalPatterns(text: string): ParsedCommand {
|
||||
// Pattern: "make [target] [color]"
|
||||
// Pattern: "add [element] to [target]"
|
||||
// Pattern: "connect [source] to [destination]"
|
||||
// etc.
|
||||
}
|
||||
|
||||
private async aiParse(text: string, context: CommandContext): Promise<ParsedCommand> {
|
||||
const prompt = `Parse this Noodl editing command:
|
||||
Command: "${text}"
|
||||
|
||||
Current selection: ${context.selection.map(n => n.type.localName).join(', ')}
|
||||
Component: ${context.component.name}
|
||||
|
||||
Output JSON with: intent, targets, modifications`;
|
||||
|
||||
const response = await this.anthropicClient.complete(prompt);
|
||||
return JSON.parse(response);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Change Generator
|
||||
|
||||
```typescript
|
||||
// packages/noodl-editor/src/editor/src/services/ai/ChangeGenerator.ts
|
||||
|
||||
class ChangeGenerator {
|
||||
// Generate actual changes from parsed command
|
||||
async generateChanges(command: ParsedCommand, context: CommandContext): Promise<Change[]> {
|
||||
const changes: Change[] = [];
|
||||
|
||||
switch (command.intent) {
|
||||
case CommandIntent.STYLE_CHANGE:
|
||||
changes.push(...this.generateStyleChanges(command, context));
|
||||
break;
|
||||
case CommandIntent.STRUCTURE_CHANGE:
|
||||
changes.push(...await this.generateStructureChanges(command, context));
|
||||
break;
|
||||
case CommandIntent.LOGIC_CHANGE:
|
||||
changes.push(...await this.generateLogicChanges(command, context));
|
||||
break;
|
||||
// ...
|
||||
}
|
||||
|
||||
return changes;
|
||||
}
|
||||
|
||||
private generateStyleChanges(command: ParsedCommand, context: CommandContext): Change[] {
|
||||
const changes: Change[] = [];
|
||||
|
||||
for (const target of command.targets) {
|
||||
const node = this.resolveTarget(target, context);
|
||||
|
||||
for (const mod of command.modifications) {
|
||||
const propertyName = this.mapToNoodlProperty(mod.property);
|
||||
const newValue = this.parseValue(mod.value, propertyName);
|
||||
|
||||
changes.push({
|
||||
type: 'property',
|
||||
target: node.id,
|
||||
before: node.parameters[propertyName],
|
||||
after: newValue,
|
||||
description: `Change ${propertyName} to ${newValue}`
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return changes;
|
||||
}
|
||||
|
||||
private async generateStructureChanges(
|
||||
command: ParsedCommand,
|
||||
context: CommandContext
|
||||
): Promise<Change[]> {
|
||||
// Use AI to generate node structure
|
||||
const prompt = `Generate Noodl node structure for:
|
||||
"${command.modifications.map(m => m.description).join(', ')}"
|
||||
|
||||
Current context: ${JSON.stringify(context.selection.map(n => n.type.localName))}
|
||||
|
||||
Output JSON array of nodes to create and connections`;
|
||||
|
||||
const response = await this.anthropicClient.complete(prompt);
|
||||
return this.parseStructureResponse(response);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 4. UI Components
|
||||
|
||||
#### Command Palette
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ 🔮 What do you want to do? [×] │
|
||||
├─────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Make the button larger and add a hover effect │ │
|
||||
│ └─────────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ Selected: Button, Text │
|
||||
│ │
|
||||
│ Recent: │
|
||||
│ • "Add loading spinner to the form" │
|
||||
│ • "Make all headers blue" │
|
||||
│ • "Connect the submit button to the API" │
|
||||
│ │
|
||||
│ Examples: │
|
||||
│ • "Wrap this in a card with shadow" │
|
||||
│ • "Add validation to all inputs" │
|
||||
│ • "Show error message when API fails" │
|
||||
│ │
|
||||
│ [Preview Changes] │
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
#### Change Preview
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ Preview Changes [×] │
|
||||
├─────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ Command: "Make the button larger and add a hover effect" │
|
||||
│ │
|
||||
│ Changes to apply: │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ ✓ Button │ │
|
||||
│ │ • width: 100px → 150px │ │
|
||||
│ │ • height: 40px → 50px │ │
|
||||
│ │ • fontSize: 14px → 16px │ │
|
||||
│ ├─────────────────────────────────────────────────────────────────┤ │
|
||||
│ │ + New: HoverState │ │
|
||||
│ │ • scale: 1.05 │ │
|
||||
│ │ • transition: 200ms │ │
|
||||
│ └─────────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ 🤖 "I'll increase the button size by 50% and add a subtle scale │
|
||||
│ effect on hover with a smooth transition." │
|
||||
│ │
|
||||
│ [Cancel] [Modify] [Apply Changes] │
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
#### Inline Command
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ Canvas │
|
||||
│ │
|
||||
│ ┌─────────────────┐ │
|
||||
│ │ [Button] │ ← Selected │
|
||||
│ │ Click me │ │
|
||||
│ └─────────────────┘ │
|
||||
│ │ │
|
||||
│ ┌──────┴──────────────────────────────────────────────┐ │
|
||||
│ │ 🔮 Make it red with rounded corners [Enter] │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### 5. Command Patterns
|
||||
|
||||
```typescript
|
||||
// packages/noodl-editor/src/editor/src/services/ai/CommandPatterns.ts
|
||||
|
||||
const COMMAND_PATTERNS: CommandPattern[] = [
|
||||
// Style patterns
|
||||
{
|
||||
pattern: /make (?:it |this |the )?(.+?) (red|blue|green|...)/i,
|
||||
intent: CommandIntent.STYLE_CHANGE,
|
||||
extract: (match) => ({
|
||||
target: match[1] || 'selection',
|
||||
property: 'backgroundColor',
|
||||
value: match[2]
|
||||
})
|
||||
},
|
||||
{
|
||||
pattern: /(?:set |change )?(?:the )?(.+?) (?:to |=) (.+)/i,
|
||||
intent: CommandIntent.STYLE_CHANGE,
|
||||
extract: (match) => ({
|
||||
property: match[1],
|
||||
value: match[2]
|
||||
})
|
||||
},
|
||||
|
||||
// Structure patterns
|
||||
{
|
||||
pattern: /add (?:a |an )?(.+?) (?:to |inside |in) (.+)/i,
|
||||
intent: CommandIntent.STRUCTURE_CHANGE,
|
||||
extract: (match) => ({
|
||||
nodeType: match[1],
|
||||
target: match[2]
|
||||
})
|
||||
},
|
||||
{
|
||||
pattern: /wrap (?:it |this |selection )?in (?:a |an )?(.+)/i,
|
||||
intent: CommandIntent.STRUCTURE_CHANGE,
|
||||
extract: (match) => ({
|
||||
action: 'wrap',
|
||||
wrapper: match[1]
|
||||
})
|
||||
},
|
||||
|
||||
// Logic patterns
|
||||
{
|
||||
pattern: /show (.+?) when (.+)/i,
|
||||
intent: CommandIntent.LOGIC_CHANGE,
|
||||
extract: (match) => ({
|
||||
action: 'conditional_show',
|
||||
target: match[1],
|
||||
condition: match[2]
|
||||
})
|
||||
},
|
||||
{
|
||||
pattern: /connect (.+?) to (.+)/i,
|
||||
intent: CommandIntent.CONNECT,
|
||||
extract: (match) => ({
|
||||
source: match[1],
|
||||
destination: match[2]
|
||||
})
|
||||
}
|
||||
];
|
||||
```
|
||||
|
||||
## Files to Create
|
||||
|
||||
1. `packages/noodl-editor/src/editor/src/services/NaturalLanguageService.ts`
|
||||
2. `packages/noodl-editor/src/editor/src/services/ai/CommandParser.ts`
|
||||
3. `packages/noodl-editor/src/editor/src/services/ai/ChangeGenerator.ts`
|
||||
4. `packages/noodl-editor/src/editor/src/services/ai/CommandPatterns.ts`
|
||||
5. `packages/noodl-core-ui/src/components/ai/CommandPalette/CommandPalette.tsx`
|
||||
6. `packages/noodl-core-ui/src/components/ai/ChangePreview/ChangePreview.tsx`
|
||||
7. `packages/noodl-core-ui/src/components/ai/InlineCommand/InlineCommand.tsx`
|
||||
|
||||
## Files to Modify
|
||||
|
||||
1. `packages/noodl-editor/src/editor/src/views/nodegrapheditor.js`
|
||||
- Add command palette trigger (Cmd+K)
|
||||
- Add inline command on selection
|
||||
- Handle change application
|
||||
|
||||
2. `packages/noodl-editor/src/editor/src/pages/EditorPage/EditorPage.tsx`
|
||||
- Add keyboard shortcut handler
|
||||
- Mount command palette
|
||||
|
||||
3. `packages/noodl-editor/src/editor/src/models/NodeGraphModel.ts`
|
||||
- Add atomic change application
|
||||
- Support undo for AI changes
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Phase 1: Command Infrastructure
|
||||
1. Create NaturalLanguageService
|
||||
2. Implement command history
|
||||
3. Set up undo/redo support
|
||||
|
||||
### Phase 2: Command Parser
|
||||
1. Create CommandParser
|
||||
2. Define local patterns
|
||||
3. Implement AI parsing
|
||||
4. Test parsing accuracy
|
||||
|
||||
### Phase 3: Change Generator
|
||||
1. Create ChangeGenerator
|
||||
2. Implement style changes
|
||||
3. Implement structure changes
|
||||
4. Implement logic changes
|
||||
|
||||
### Phase 4: UI - Command Palette
|
||||
1. Create CommandPalette component
|
||||
2. Add keyboard shortcut
|
||||
3. Show recent/examples
|
||||
4. Handle input
|
||||
|
||||
### Phase 5: UI - Change Preview
|
||||
1. Create ChangePreview component
|
||||
2. Show before/after
|
||||
3. Add explanation
|
||||
4. Handle apply/cancel
|
||||
|
||||
### Phase 6: UI - Inline Command
|
||||
1. Create InlineCommand component
|
||||
2. Position near selection
|
||||
3. Handle quick commands
|
||||
|
||||
### Phase 7: Learning & Improvement
|
||||
1. Track command success
|
||||
2. Record corrections
|
||||
3. Improve pattern matching
|
||||
|
||||
## Testing Checklist
|
||||
|
||||
- [ ] Style commands work correctly
|
||||
- [ ] Structure commands create nodes
|
||||
- [ ] Logic commands set up conditions
|
||||
- [ ] Preview shows accurate changes
|
||||
- [ ] Apply actually makes changes
|
||||
- [ ] Undo reverts changes
|
||||
- [ ] Keyboard shortcuts work
|
||||
- [ ] Recent commands saved
|
||||
- [ ] Error handling graceful
|
||||
- [ ] Complex commands work
|
||||
- [ ] Multi-target commands work
|
||||
|
||||
## Dependencies
|
||||
|
||||
- AI-001 (AI Project Scaffolding) - for AnthropicClient
|
||||
- AI-002 (Component Suggestions) - for context analysis
|
||||
|
||||
## Blocked By
|
||||
|
||||
- AI-001
|
||||
|
||||
## Blocks
|
||||
|
||||
- AI-004 (AI Design Assistance)
|
||||
|
||||
## Estimated Effort
|
||||
|
||||
- Command service: 4-5 hours
|
||||
- Command parser: 5-6 hours
|
||||
- Change generator: 6-8 hours
|
||||
- UI command palette: 4-5 hours
|
||||
- UI change preview: 4-5 hours
|
||||
- UI inline command: 3-4 hours
|
||||
- Testing & refinement: 4-5 hours
|
||||
- **Total: 30-38 hours**
|
||||
|
||||
## Success Criteria
|
||||
|
||||
1. Natural language commands understood
|
||||
2. Preview shows accurate changes
|
||||
3. Changes applied correctly
|
||||
4. Undo/redo works
|
||||
5. < 3 second response time
|
||||
6. 80%+ command success rate
|
||||
|
||||
## Command Examples
|
||||
|
||||
### Style Changes
|
||||
- "Make it red"
|
||||
- "Add a shadow"
|
||||
- "Increase font size to 18"
|
||||
- "Round the corners"
|
||||
- "Make all buttons blue"
|
||||
|
||||
### Structure Changes
|
||||
- "Add a header"
|
||||
- "Wrap in a card"
|
||||
- "Add a loading spinner"
|
||||
- "Create a sidebar"
|
||||
- "Split into two columns"
|
||||
|
||||
### Logic Changes
|
||||
- "Show loading while fetching"
|
||||
- "Hide when empty"
|
||||
- "Disable until form is valid"
|
||||
- "Navigate to home on click"
|
||||
- "Show error message on failure"
|
||||
|
||||
### Data Changes
|
||||
- "Sort by date"
|
||||
- "Filter completed items"
|
||||
- "Group by category"
|
||||
- "Limit to 10 items"
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
- Voice input
|
||||
- Multi-language support
|
||||
- Command macros
|
||||
- Batch changes
|
||||
- Command sharing
|
||||
- Context-aware autocomplete
|
||||
@@ -0,0 +1,681 @@
|
||||
# AI-004: AI Design Assistance
|
||||
|
||||
## Overview
|
||||
|
||||
Provide AI-powered design feedback and improvements. Analyze components for design issues (accessibility, consistency, spacing) and suggest or auto-apply fixes. Transform rough layouts into polished designs.
|
||||
|
||||
## Context
|
||||
|
||||
Many Noodl users are developers or designers who may not have deep expertise in both areas. Common issues include:
|
||||
- Inconsistent spacing and alignment
|
||||
- Accessibility problems (contrast, touch targets)
|
||||
- Missing hover/focus states
|
||||
- Unbalanced layouts
|
||||
- Poor color combinations
|
||||
|
||||
AI Design Assistance provides:
|
||||
- Automated design review
|
||||
- One-click fixes for common issues
|
||||
- Style consistency enforcement
|
||||
- Accessibility compliance checking
|
||||
- Layout optimization suggestions
|
||||
|
||||
## Requirements
|
||||
|
||||
### Functional Requirements
|
||||
|
||||
1. **Design Analysis**
|
||||
- Scan component/page for issues
|
||||
- Categorize by severity (error, warning, info)
|
||||
- Group by type (spacing, color, typography, etc.)
|
||||
- Provide explanations
|
||||
|
||||
2. **Issue Categories**
|
||||
- **Accessibility**: Contrast, touch targets, labels
|
||||
- **Consistency**: Spacing, colors, typography
|
||||
- **Layout**: Alignment, balance, whitespace
|
||||
- **Interaction**: Hover, focus, active states
|
||||
- **Responsiveness**: Breakpoint issues
|
||||
|
||||
3. **Fix Application**
|
||||
- One-click fix for individual issues
|
||||
- "Fix all" for category
|
||||
- Preview before applying
|
||||
- Explain what was fixed
|
||||
|
||||
4. **Design Improvement**
|
||||
- "Polish this" command
|
||||
- Transform rough layouts
|
||||
- Suggest design alternatives
|
||||
- Apply consistent styling
|
||||
|
||||
5. **Design System Enforcement**
|
||||
- Check against project styles
|
||||
- Suggest using existing styles
|
||||
- Identify one-off values
|
||||
- Propose new styles
|
||||
|
||||
### Non-Functional Requirements
|
||||
|
||||
- Analysis completes in < 5 seconds
|
||||
- Fixes don't break functionality
|
||||
- Respects existing design intent
|
||||
- Works with any component structure
|
||||
|
||||
## Technical Approach
|
||||
|
||||
### 1. Design Analysis Service
|
||||
|
||||
```typescript
|
||||
// packages/noodl-editor/src/editor/src/services/DesignAnalysisService.ts
|
||||
|
||||
interface DesignIssue {
|
||||
id: string;
|
||||
type: IssueType;
|
||||
severity: 'error' | 'warning' | 'info';
|
||||
category: IssueCategory;
|
||||
node: NodeGraphNode;
|
||||
property?: string;
|
||||
message: string;
|
||||
explanation: string;
|
||||
fix?: DesignFix;
|
||||
}
|
||||
|
||||
enum IssueCategory {
|
||||
ACCESSIBILITY = 'accessibility',
|
||||
CONSISTENCY = 'consistency',
|
||||
LAYOUT = 'layout',
|
||||
INTERACTION = 'interaction',
|
||||
RESPONSIVENESS = 'responsiveness'
|
||||
}
|
||||
|
||||
interface DesignFix {
|
||||
description: string;
|
||||
changes: PropertyChange[];
|
||||
preview?: string;
|
||||
}
|
||||
|
||||
class DesignAnalysisService {
|
||||
private static instance: DesignAnalysisService;
|
||||
private analyzers: DesignAnalyzer[] = [];
|
||||
|
||||
// Analysis
|
||||
async analyzeComponent(component: ComponentModel): Promise<DesignIssue[]>;
|
||||
async analyzePage(page: ComponentModel): Promise<DesignIssue[]>;
|
||||
async analyzeProject(project: ProjectModel): Promise<DesignIssue[]>;
|
||||
|
||||
// Fixes
|
||||
async applyFix(issue: DesignIssue): Promise<void>;
|
||||
async applyAllFixes(issues: DesignIssue[]): Promise<void>;
|
||||
async previewFix(issue: DesignIssue): Promise<FixPreview>;
|
||||
|
||||
// Polish
|
||||
async polishComponent(component: ComponentModel): Promise<PolishResult>;
|
||||
async suggestImprovements(component: ComponentModel): Promise<Improvement[]>;
|
||||
|
||||
// Design system
|
||||
async checkDesignSystem(component: ComponentModel): Promise<DesignSystemIssue[]>;
|
||||
async suggestStyles(component: ComponentModel): Promise<StyleSuggestion[]>;
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Design Analyzers
|
||||
|
||||
```typescript
|
||||
// packages/noodl-editor/src/editor/src/services/ai/analyzers/
|
||||
|
||||
// Base analyzer
|
||||
interface DesignAnalyzer {
|
||||
name: string;
|
||||
category: IssueCategory;
|
||||
analyze(component: ComponentModel): DesignIssue[];
|
||||
}
|
||||
|
||||
// Accessibility Analyzer
|
||||
class AccessibilityAnalyzer implements DesignAnalyzer {
|
||||
name = 'Accessibility';
|
||||
category = IssueCategory.ACCESSIBILITY;
|
||||
|
||||
analyze(component: ComponentModel): DesignIssue[] {
|
||||
const issues: DesignIssue[] = [];
|
||||
|
||||
component.forEachNode(node => {
|
||||
// Check contrast
|
||||
if (this.hasTextAndBackground(node)) {
|
||||
const contrast = this.calculateContrast(
|
||||
node.parameters.color,
|
||||
node.parameters.backgroundColor
|
||||
);
|
||||
if (contrast < 4.5) {
|
||||
issues.push({
|
||||
type: 'low-contrast',
|
||||
severity: contrast < 3 ? 'error' : 'warning',
|
||||
category: IssueCategory.ACCESSIBILITY,
|
||||
node,
|
||||
message: `Low color contrast (${contrast.toFixed(1)}:1)`,
|
||||
explanation: 'WCAG requires 4.5:1 for normal text',
|
||||
fix: {
|
||||
description: 'Adjust colors for better contrast',
|
||||
changes: this.suggestContrastFix(node)
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Check touch targets
|
||||
if (this.isInteractive(node)) {
|
||||
const size = this.getSize(node);
|
||||
if (size.width < 44 || size.height < 44) {
|
||||
issues.push({
|
||||
type: 'small-touch-target',
|
||||
severity: 'warning',
|
||||
category: IssueCategory.ACCESSIBILITY,
|
||||
node,
|
||||
message: 'Touch target too small',
|
||||
explanation: 'Minimum 44x44px recommended for touch',
|
||||
fix: {
|
||||
description: 'Increase size to 44x44px minimum',
|
||||
changes: [
|
||||
{ property: 'width', value: Math.max(size.width, 44) },
|
||||
{ property: 'height', value: Math.max(size.height, 44) }
|
||||
]
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Check labels
|
||||
if (this.isFormInput(node) && !this.hasLabel(node)) {
|
||||
issues.push({
|
||||
type: 'missing-label',
|
||||
severity: 'error',
|
||||
category: IssueCategory.ACCESSIBILITY,
|
||||
node,
|
||||
message: 'Form input missing label',
|
||||
explanation: 'Screen readers need labels to identify inputs'
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
return issues;
|
||||
}
|
||||
}
|
||||
|
||||
// Consistency Analyzer
|
||||
class ConsistencyAnalyzer implements DesignAnalyzer {
|
||||
name = 'Consistency';
|
||||
category = IssueCategory.CONSISTENCY;
|
||||
|
||||
analyze(component: ComponentModel): DesignIssue[] {
|
||||
const issues: DesignIssue[] = [];
|
||||
|
||||
// Collect all values
|
||||
const spacings = this.collectSpacings(component);
|
||||
const colors = this.collectColors(component);
|
||||
const fontSizes = this.collectFontSizes(component);
|
||||
|
||||
// Check for one-offs
|
||||
const spacingOneOffs = this.findOneOffs(spacings, SPACING_SCALE);
|
||||
const colorOneOffs = this.findOneOffs(colors, component.colorStyles);
|
||||
const fontOneOffs = this.findOneOffs(fontSizes, FONT_SCALE);
|
||||
|
||||
// Report issues
|
||||
for (const oneOff of spacingOneOffs) {
|
||||
issues.push({
|
||||
type: 'inconsistent-spacing',
|
||||
severity: 'info',
|
||||
category: IssueCategory.CONSISTENCY,
|
||||
node: oneOff.node,
|
||||
property: oneOff.property,
|
||||
message: `Non-standard spacing: ${oneOff.value}`,
|
||||
explanation: 'Consider using a standard spacing value',
|
||||
fix: {
|
||||
description: `Change to ${oneOff.suggestion}`,
|
||||
changes: [{ property: oneOff.property, value: oneOff.suggestion }]
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
return issues;
|
||||
}
|
||||
}
|
||||
|
||||
// Layout Analyzer
|
||||
class LayoutAnalyzer implements DesignAnalyzer {
|
||||
name = 'Layout';
|
||||
category = IssueCategory.LAYOUT;
|
||||
|
||||
analyze(component: ComponentModel): DesignIssue[] {
|
||||
const issues: DesignIssue[] = [];
|
||||
|
||||
// Check alignment
|
||||
const alignmentIssues = this.checkAlignment(component);
|
||||
issues.push(...alignmentIssues);
|
||||
|
||||
// Check whitespace balance
|
||||
const whitespaceIssues = this.checkWhitespace(component);
|
||||
issues.push(...whitespaceIssues);
|
||||
|
||||
// Check visual hierarchy
|
||||
const hierarchyIssues = this.checkHierarchy(component);
|
||||
issues.push(...hierarchyIssues);
|
||||
|
||||
return issues;
|
||||
}
|
||||
}
|
||||
|
||||
// Interaction Analyzer
|
||||
class InteractionAnalyzer implements DesignAnalyzer {
|
||||
name = 'Interaction';
|
||||
category = IssueCategory.INTERACTION;
|
||||
|
||||
analyze(component: ComponentModel): DesignIssue[] {
|
||||
const issues: DesignIssue[] = [];
|
||||
|
||||
component.forEachNode(node => {
|
||||
if (this.isInteractive(node)) {
|
||||
// Check hover state
|
||||
if (!this.hasHoverState(node)) {
|
||||
issues.push({
|
||||
type: 'missing-hover',
|
||||
severity: 'warning',
|
||||
category: IssueCategory.INTERACTION,
|
||||
node,
|
||||
message: 'Missing hover state',
|
||||
explanation: 'Interactive elements should have hover feedback',
|
||||
fix: {
|
||||
description: 'Add subtle hover effect',
|
||||
changes: this.generateHoverState(node)
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// Check focus state
|
||||
if (!this.hasFocusState(node)) {
|
||||
issues.push({
|
||||
type: 'missing-focus',
|
||||
severity: 'error',
|
||||
category: IssueCategory.INTERACTION,
|
||||
node,
|
||||
message: 'Missing focus state',
|
||||
explanation: 'Keyboard users need visible focus indicators'
|
||||
});
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
return issues;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 3. AI Polish Engine
|
||||
|
||||
```typescript
|
||||
// packages/noodl-editor/src/editor/src/services/ai/PolishEngine.ts
|
||||
|
||||
interface PolishResult {
|
||||
before: ComponentSnapshot;
|
||||
after: ComponentSnapshot;
|
||||
changes: Change[];
|
||||
explanation: string;
|
||||
}
|
||||
|
||||
class PolishEngine {
|
||||
async polishComponent(component: ComponentModel): Promise<PolishResult> {
|
||||
// 1. Analyze current state
|
||||
const issues = await DesignAnalysisService.instance.analyzeComponent(component);
|
||||
|
||||
// 2. Apply automatic fixes
|
||||
const autoFixable = issues.filter(i => i.fix && i.severity !== 'error');
|
||||
for (const issue of autoFixable) {
|
||||
await this.applyFix(issue);
|
||||
}
|
||||
|
||||
// 3. Use AI for creative improvements
|
||||
const aiImprovements = await this.getAiImprovements(component);
|
||||
|
||||
// 4. Apply AI suggestions
|
||||
for (const improvement of aiImprovements) {
|
||||
await this.applyImprovement(improvement);
|
||||
}
|
||||
|
||||
return {
|
||||
before: this.originalSnapshot,
|
||||
after: this.currentSnapshot,
|
||||
changes: this.recordedChanges,
|
||||
explanation: this.generateExplanation()
|
||||
};
|
||||
}
|
||||
|
||||
private async getAiImprovements(component: ComponentModel): Promise<Improvement[]> {
|
||||
const prompt = `Analyze this Noodl component and suggest design improvements:
|
||||
|
||||
Component structure:
|
||||
${this.serializeComponent(component)}
|
||||
|
||||
Current styles:
|
||||
${this.serializeStyles(component)}
|
||||
|
||||
Suggest improvements for:
|
||||
1. Visual hierarchy
|
||||
2. Whitespace and breathing room
|
||||
3. Color harmony
|
||||
4. Typography refinement
|
||||
5. Micro-interactions
|
||||
|
||||
Output JSON array of improvements with property changes.`;
|
||||
|
||||
const response = await this.anthropicClient.complete(prompt);
|
||||
return JSON.parse(response);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 4. UI Components
|
||||
|
||||
#### Design Review Panel
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ Design Review [×] │
|
||||
├─────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ 📊 Overview │
|
||||
│ ┌─────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ 🔴 2 Errors 🟡 5 Warnings 🔵 3 Info │ │
|
||||
│ └─────────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ 🔴 ERRORS [Fix All (2)] │
|
||||
│ ┌─────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ ♿ Low color contrast on "Submit" button [Fix] │ │
|
||||
│ │ Contrast ratio 2.1:1, needs 4.5:1 │ │
|
||||
│ ├─────────────────────────────────────────────────────────────────┤ │
|
||||
│ │ ♿ Missing label on email input [Fix] │ │
|
||||
│ │ Screen readers cannot identify this input │ │
|
||||
│ └─────────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ 🟡 WARNINGS [Fix All (5)] │
|
||||
│ ┌─────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ 📐 Inconsistent spacing (12px vs 16px scale) [Fix] │ │
|
||||
│ │ 👆 Touch target too small (32x32px) [Fix] │ │
|
||||
│ │ ✨ Missing hover state on buttons [Fix] │ │
|
||||
│ │ ... │ │
|
||||
│ └─────────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ [Analyze Again] [✨ Polish All] │
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
#### Polish Preview
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ ✨ Polish Preview [×] │
|
||||
├─────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ BEFORE AFTER │
|
||||
│ ┌────────────────────────┐ ┌────────────────────────┐ │
|
||||
│ │ ┌──────────────────┐ │ │ ┌──────────────────┐ │ │
|
||||
│ │ │ Cramped card │ │ │ │ │ │ │
|
||||
│ │ │ No shadow │ │ → │ │ Polished card │ │ │
|
||||
│ │ │ Basic button │ │ │ │ with shadow │ │ │
|
||||
│ │ └──────────────────┘ │ │ │ and spacing │ │ │
|
||||
│ │ │ │ └──────────────────┘ │ │
|
||||
│ └────────────────────────┘ └────────────────────────┘ │
|
||||
│ │
|
||||
│ Changes Applied: │
|
||||
│ • Added 24px padding to card │
|
||||
│ • Added subtle shadow (0 2px 8px rgba(0,0,0,0.1)) │
|
||||
│ • Increased button padding (12px 24px) │
|
||||
│ • Added hover state with 0.95 scale │
|
||||
│ • Adjusted border radius to 12px │
|
||||
│ │
|
||||
│ [Revert] [Apply Polish] │
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### 5. Design Rules Engine
|
||||
|
||||
```typescript
|
||||
// packages/noodl-editor/src/editor/src/services/ai/DesignRules.ts
|
||||
|
||||
interface DesignRule {
|
||||
id: string;
|
||||
name: string;
|
||||
description: string;
|
||||
category: IssueCategory;
|
||||
severity: 'error' | 'warning' | 'info';
|
||||
check: (node: NodeGraphNode, context: DesignContext) => RuleViolation | null;
|
||||
fix?: (violation: RuleViolation) => PropertyChange[];
|
||||
}
|
||||
|
||||
const DESIGN_RULES: DesignRule[] = [
|
||||
// Accessibility
|
||||
{
|
||||
id: 'min-contrast',
|
||||
name: 'Minimum Color Contrast',
|
||||
description: 'Text must have sufficient contrast with background',
|
||||
category: IssueCategory.ACCESSIBILITY,
|
||||
severity: 'error',
|
||||
check: (node, ctx) => {
|
||||
if (!hasTextAndBackground(node)) return null;
|
||||
const contrast = calculateContrast(node.parameters.color, node.parameters.backgroundColor);
|
||||
if (contrast < 4.5) {
|
||||
return { node, contrast, required: 4.5 };
|
||||
}
|
||||
return null;
|
||||
},
|
||||
fix: (violation) => suggestContrastFix(violation.node, violation.required)
|
||||
},
|
||||
|
||||
{
|
||||
id: 'min-touch-target',
|
||||
name: 'Minimum Touch Target Size',
|
||||
description: 'Interactive elements must be at least 44x44px',
|
||||
category: IssueCategory.ACCESSIBILITY,
|
||||
severity: 'warning',
|
||||
check: (node) => {
|
||||
if (!isInteractive(node)) return null;
|
||||
const size = getSize(node);
|
||||
if (size.width < 44 || size.height < 44) {
|
||||
return { node, size, required: { width: 44, height: 44 } };
|
||||
}
|
||||
return null;
|
||||
},
|
||||
fix: (violation) => [
|
||||
{ property: 'width', value: Math.max(violation.size.width, 44) },
|
||||
{ property: 'height', value: Math.max(violation.size.height, 44) }
|
||||
]
|
||||
},
|
||||
|
||||
// Consistency
|
||||
{
|
||||
id: 'spacing-scale',
|
||||
name: 'Use Spacing Scale',
|
||||
description: 'Spacing should follow the design system scale',
|
||||
category: IssueCategory.CONSISTENCY,
|
||||
severity: 'info',
|
||||
check: (node) => {
|
||||
const spacing = getSpacingValues(node);
|
||||
const nonStandard = spacing.filter(s => !SPACING_SCALE.includes(s));
|
||||
if (nonStandard.length > 0) {
|
||||
return { node, nonStandard, scale: SPACING_SCALE };
|
||||
}
|
||||
return null;
|
||||
},
|
||||
fix: (violation) => violation.nonStandard.map(s => ({
|
||||
property: s.property,
|
||||
value: findClosest(s.value, SPACING_SCALE)
|
||||
}))
|
||||
},
|
||||
|
||||
// Interaction
|
||||
{
|
||||
id: 'hover-state',
|
||||
name: 'Interactive Hover State',
|
||||
description: 'Interactive elements should have hover feedback',
|
||||
category: IssueCategory.INTERACTION,
|
||||
severity: 'warning',
|
||||
check: (node) => {
|
||||
if (isInteractive(node) && !hasHoverState(node)) {
|
||||
return { node };
|
||||
}
|
||||
return null;
|
||||
},
|
||||
fix: (violation) => generateDefaultHoverState(violation.node)
|
||||
},
|
||||
|
||||
// ... more rules
|
||||
];
|
||||
```
|
||||
|
||||
## Files to Create
|
||||
|
||||
1. `packages/noodl-editor/src/editor/src/services/DesignAnalysisService.ts`
|
||||
2. `packages/noodl-editor/src/editor/src/services/ai/analyzers/AccessibilityAnalyzer.ts`
|
||||
3. `packages/noodl-editor/src/editor/src/services/ai/analyzers/ConsistencyAnalyzer.ts`
|
||||
4. `packages/noodl-editor/src/editor/src/services/ai/analyzers/LayoutAnalyzer.ts`
|
||||
5. `packages/noodl-editor/src/editor/src/services/ai/analyzers/InteractionAnalyzer.ts`
|
||||
6. `packages/noodl-editor/src/editor/src/services/ai/PolishEngine.ts`
|
||||
7. `packages/noodl-editor/src/editor/src/services/ai/DesignRules.ts`
|
||||
8. `packages/noodl-core-ui/src/components/ai/DesignReviewPanel/DesignReviewPanel.tsx`
|
||||
9. `packages/noodl-core-ui/src/components/ai/DesignIssueCard/DesignIssueCard.tsx`
|
||||
10. `packages/noodl-core-ui/src/components/ai/PolishPreview/PolishPreview.tsx`
|
||||
|
||||
## Files to Modify
|
||||
|
||||
1. `packages/noodl-editor/src/editor/src/pages/EditorPage/EditorPage.tsx`
|
||||
- Add Design Review panel toggle
|
||||
- Add menu option
|
||||
|
||||
2. `packages/noodl-editor/src/editor/src/views/panels/propertiespanel/`
|
||||
- Add issue indicators on properties
|
||||
- Quick fix buttons
|
||||
|
||||
3. `packages/noodl-editor/src/editor/src/views/nodegrapheditor.js`
|
||||
- Highlight nodes with issues
|
||||
- Add "Polish" context menu
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Phase 1: Analysis Infrastructure
|
||||
1. Create DesignAnalysisService
|
||||
2. Define issue types and categories
|
||||
3. Create analyzer base class
|
||||
4. Implement fix application
|
||||
|
||||
### Phase 2: Core Analyzers
|
||||
1. Implement AccessibilityAnalyzer
|
||||
2. Implement ConsistencyAnalyzer
|
||||
3. Implement LayoutAnalyzer
|
||||
4. Implement InteractionAnalyzer
|
||||
|
||||
### Phase 3: Polish Engine
|
||||
1. Create PolishEngine
|
||||
2. Implement auto-fix application
|
||||
3. Add AI improvement suggestions
|
||||
4. Generate explanations
|
||||
|
||||
### Phase 4: UI - Review Panel
|
||||
1. Create DesignReviewPanel
|
||||
2. Create DesignIssueCard
|
||||
3. Group issues by category
|
||||
4. Add fix buttons
|
||||
|
||||
### Phase 5: UI - Polish Preview
|
||||
1. Create PolishPreview
|
||||
2. Show before/after
|
||||
3. List changes
|
||||
4. Apply/revert actions
|
||||
|
||||
### Phase 6: Integration
|
||||
1. Add to editor menus
|
||||
2. Highlight issues on canvas
|
||||
3. Add keyboard shortcuts
|
||||
|
||||
## Testing Checklist
|
||||
|
||||
- [ ] Accessibility issues detected
|
||||
- [ ] Contrast calculation accurate
|
||||
- [ ] Touch target check works
|
||||
- [ ] Consistency issues found
|
||||
- [ ] Fixes don't break layout
|
||||
- [ ] Polish improves design
|
||||
- [ ] Preview accurate
|
||||
- [ ] Undo works
|
||||
- [ ] Performance acceptable
|
||||
- [ ] Works on all component types
|
||||
|
||||
## Dependencies
|
||||
|
||||
- AI-001 (AI Project Scaffolding) - for AnthropicClient
|
||||
- AI-003 (Natural Language Editing) - for change application
|
||||
|
||||
## Blocked By
|
||||
|
||||
- AI-001
|
||||
|
||||
## Blocks
|
||||
|
||||
- None (final task in AI series)
|
||||
|
||||
## Estimated Effort
|
||||
|
||||
- Analysis service: 4-5 hours
|
||||
- Accessibility analyzer: 4-5 hours
|
||||
- Consistency analyzer: 3-4 hours
|
||||
- Layout analyzer: 3-4 hours
|
||||
- Interaction analyzer: 3-4 hours
|
||||
- Polish engine: 5-6 hours
|
||||
- UI review panel: 4-5 hours
|
||||
- UI polish preview: 3-4 hours
|
||||
- Integration: 3-4 hours
|
||||
- **Total: 32-41 hours**
|
||||
|
||||
## Success Criteria
|
||||
|
||||
1. Issues detected accurately
|
||||
2. Fixes don't break functionality
|
||||
3. Polish improves design quality
|
||||
4. Accessibility issues caught
|
||||
5. One-click fixes work
|
||||
6. Preview shows accurate changes
|
||||
|
||||
## Design Rules Categories
|
||||
|
||||
### Accessibility (WCAG)
|
||||
- Color contrast (4.5:1 text, 3:1 large)
|
||||
- Touch targets (44x44px)
|
||||
- Focus indicators
|
||||
- Label associations
|
||||
- Alt text for images
|
||||
|
||||
### Consistency
|
||||
- Spacing scale adherence
|
||||
- Color from palette
|
||||
- Typography scale
|
||||
- Border radius consistency
|
||||
- Shadow consistency
|
||||
|
||||
### Layout
|
||||
- Alignment on grid
|
||||
- Balanced whitespace
|
||||
- Visual hierarchy
|
||||
- Content grouping
|
||||
|
||||
### Interaction
|
||||
- Hover states
|
||||
- Focus states
|
||||
- Active states
|
||||
- Loading states
|
||||
- Error states
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
- Design system integration
|
||||
- Custom rule creation
|
||||
- Team design standards
|
||||
- A/B testing suggestions
|
||||
- Animation review
|
||||
- Performance impact analysis
|
||||
@@ -0,0 +1,425 @@
|
||||
# AI Series: AI-Powered Development
|
||||
|
||||
## Overview
|
||||
|
||||
The AI series transforms OpenNoodl from a visual development tool into an intelligent development partner. Users can describe what they want to build, receive contextual suggestions, edit with natural language, and get automatic design feedback—all powered by Claude AI.
|
||||
|
||||
## Target Environment
|
||||
|
||||
- **Editor**: React 19 version only
|
||||
- **Runtime**: Not affected
|
||||
- **API**: Anthropic Claude API
|
||||
- **Fallback**: Graceful degradation without API
|
||||
|
||||
## Task Dependency Graph
|
||||
|
||||
```
|
||||
AI-001 (Project Scaffolding)
|
||||
│
|
||||
├──────────────────────────┐
|
||||
↓ ↓
|
||||
AI-002 (Suggestions) AI-003 (NL Editing)
|
||||
│ │
|
||||
└──────────┬───────────────┘
|
||||
↓
|
||||
AI-004 (Design Assistance)
|
||||
```
|
||||
|
||||
## Task Summary
|
||||
|
||||
| Task ID | Name | Est. Hours | Priority |
|
||||
|---------|------|------------|----------|
|
||||
| AI-001 | AI Project Scaffolding | 32-41 | Critical |
|
||||
| AI-002 | AI Component Suggestions | 27-34 | High |
|
||||
| AI-003 | Natural Language Editing | 30-38 | High |
|
||||
| AI-004 | AI Design Assistance | 32-41 | Medium |
|
||||
|
||||
**Total Estimated: 121-154 hours**
|
||||
|
||||
## Implementation Order
|
||||
|
||||
### Phase 1: Foundation (Weeks 1-3)
|
||||
1. **AI-001** - Project scaffolding with AI
|
||||
- Establishes Anthropic API integration
|
||||
- Creates core AI services
|
||||
- Delivers immediate user value
|
||||
|
||||
### Phase 2: In-Editor Intelligence (Weeks 4-6)
|
||||
2. **AI-002** - Component suggestions
|
||||
- Context-aware recommendations
|
||||
- Pattern library foundation
|
||||
3. **AI-003** - Natural language editing
|
||||
- Command palette for AI edits
|
||||
- Change preview and application
|
||||
|
||||
### Phase 3: Design Quality (Weeks 7-8)
|
||||
4. **AI-004** - Design assistance
|
||||
- Automated design review
|
||||
- Polish and improvements
|
||||
|
||||
## Existing Infrastructure
|
||||
|
||||
### AiAssistantModel
|
||||
|
||||
```typescript
|
||||
// Current AI node system
|
||||
class AiAssistantModel {
|
||||
templates: AiTemplate[]; // REST, Function, Form Validation, etc.
|
||||
|
||||
createNode(templateId, parentModel, pos);
|
||||
createContext(node);
|
||||
send(context);
|
||||
}
|
||||
```
|
||||
|
||||
### AI Templates
|
||||
|
||||
```typescript
|
||||
docsTemplates = [
|
||||
{ label: 'REST API', template: 'rest' },
|
||||
{ label: 'Form Validation', template: 'function-form-validation' },
|
||||
{ label: 'AI Function', template: 'function' },
|
||||
{ label: 'Write to database', template: 'function-crud' }
|
||||
];
|
||||
```
|
||||
|
||||
### Template Registry
|
||||
|
||||
```typescript
|
||||
// Project template system
|
||||
templateRegistry.list({}); // List available templates
|
||||
templateRegistry.download({ templateUrl }); // Download template
|
||||
```
|
||||
|
||||
### LocalProjectsModel
|
||||
|
||||
```typescript
|
||||
// Project creation
|
||||
LocalProjectsModel.newProject(callback, {
|
||||
name,
|
||||
path,
|
||||
projectTemplate
|
||||
});
|
||||
```
|
||||
|
||||
## New Architecture
|
||||
|
||||
### Core AI Services
|
||||
|
||||
```
|
||||
packages/noodl-editor/src/editor/src/services/
|
||||
├── ai/
|
||||
│ ├── AnthropicClient.ts # Claude API wrapper
|
||||
│ ├── prompts/ # Prompt templates
|
||||
│ │ ├── scaffolding.ts
|
||||
│ │ ├── suggestions.ts
|
||||
│ │ └── editing.ts
|
||||
│ ├── ContextAnalyzer.ts # Component analysis
|
||||
│ ├── PatternLibrary.ts # Known patterns
|
||||
│ ├── CommandParser.ts # NL command parsing
|
||||
│ ├── ChangeGenerator.ts # Generate changes
|
||||
│ └── analyzers/ # Design analyzers
|
||||
│ ├── AccessibilityAnalyzer.ts
|
||||
│ ├── ConsistencyAnalyzer.ts
|
||||
│ └── ...
|
||||
├── AiScaffoldingService.ts
|
||||
├── AiSuggestionService.ts
|
||||
├── NaturalLanguageService.ts
|
||||
└── DesignAnalysisService.ts
|
||||
```
|
||||
|
||||
### Service Hierarchy
|
||||
|
||||
| Service | Purpose |
|
||||
|---------|---------|
|
||||
| AnthropicClient | Claude API communication |
|
||||
| AiScaffoldingService | Project generation |
|
||||
| AiSuggestionService | Context-aware suggestions |
|
||||
| NaturalLanguageService | Command parsing & execution |
|
||||
| DesignAnalysisService | Design review & fixes |
|
||||
|
||||
## API Integration
|
||||
|
||||
### Anthropic Client
|
||||
|
||||
```typescript
|
||||
class AnthropicClient {
|
||||
private apiKey: string;
|
||||
private model = 'claude-sonnet-4-20250514';
|
||||
|
||||
async complete(prompt: string, options?: CompletionOptions): Promise<string>;
|
||||
async chat(messages: Message[]): Promise<Message>;
|
||||
async stream(prompt: string, onChunk: (chunk: string) => void): Promise<void>;
|
||||
}
|
||||
```
|
||||
|
||||
### API Key Management
|
||||
|
||||
```typescript
|
||||
// Settings storage
|
||||
interface AiSettings {
|
||||
apiKey: string; // Stored securely
|
||||
enabled: boolean;
|
||||
features: {
|
||||
scaffolding: boolean;
|
||||
suggestions: boolean;
|
||||
naturalLanguage: boolean;
|
||||
designAssistance: boolean;
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
## Key User Flows
|
||||
|
||||
### 1. Create Project from Description
|
||||
|
||||
```
|
||||
User opens "New Project"
|
||||
↓
|
||||
Selects "Describe your project"
|
||||
↓
|
||||
Types: "A task management app with kanban board"
|
||||
↓
|
||||
AI generates scaffold
|
||||
↓
|
||||
User previews & refines via chat
|
||||
↓
|
||||
Creates actual project
|
||||
```
|
||||
|
||||
### 2. Get Suggestions While Building
|
||||
|
||||
```
|
||||
User adds TextInput node
|
||||
↓
|
||||
System detects incomplete form pattern
|
||||
↓
|
||||
Shows suggestion: "Add form validation?"
|
||||
↓
|
||||
User clicks "Apply"
|
||||
↓
|
||||
Validation nodes added automatically
|
||||
```
|
||||
|
||||
### 3. Edit with Natural Language
|
||||
|
||||
```
|
||||
User selects Button node
|
||||
↓
|
||||
Presses Cmd+K
|
||||
↓
|
||||
Types: "Make it larger with a hover effect"
|
||||
↓
|
||||
Preview shows changes
|
||||
↓
|
||||
User clicks "Apply"
|
||||
```
|
||||
|
||||
### 4. Design Review & Polish
|
||||
|
||||
```
|
||||
User opens Design Review panel
|
||||
↓
|
||||
AI analyzes component
|
||||
↓
|
||||
Shows: "2 accessibility issues, 3 warnings"
|
||||
↓
|
||||
User clicks "Fix All" or "Polish"
|
||||
↓
|
||||
Changes applied automatically
|
||||
```
|
||||
|
||||
## UI Components to Create
|
||||
|
||||
| Component | Package | Purpose |
|
||||
|-----------|---------|---------|
|
||||
| AiProjectModal | noodl-core-ui | Project scaffolding UI |
|
||||
| ScaffoldPreview | noodl-core-ui | Preview generated structure |
|
||||
| SuggestionHint | noodl-core-ui | Inline suggestion display |
|
||||
| SuggestionPanel | noodl-core-ui | Full suggestions list |
|
||||
| CommandPalette | noodl-core-ui | NL command input |
|
||||
| ChangePreview | noodl-core-ui | Show pending changes |
|
||||
| DesignReviewPanel | noodl-core-ui | Design issues list |
|
||||
| PolishPreview | noodl-core-ui | Before/after comparison |
|
||||
|
||||
## Prompt Engineering
|
||||
|
||||
### System Prompts
|
||||
|
||||
```typescript
|
||||
// Scaffolding
|
||||
const SCAFFOLD_SYSTEM = `You are an expert Noodl application architect.
|
||||
Generate detailed project scaffolds for visual low-code applications.
|
||||
Consider: UX flow, data management, reusability, performance.`;
|
||||
|
||||
// Suggestions
|
||||
const SUGGESTION_SYSTEM = `You analyze Noodl components and suggest
|
||||
improvements. Focus on: pattern completion, best practices,
|
||||
common UI patterns, data handling.`;
|
||||
|
||||
// Natural Language
|
||||
const NL_SYSTEM = `You parse natural language commands for editing
|
||||
Noodl visual components. Output structured changes that can be
|
||||
applied to the node graph.`;
|
||||
|
||||
// Design
|
||||
const DESIGN_SYSTEM = `You are a design expert analyzing Noodl
|
||||
components for accessibility, consistency, and visual quality.
|
||||
Suggest concrete property changes.`;
|
||||
```
|
||||
|
||||
### Context Serialization
|
||||
|
||||
```typescript
|
||||
// Serialize component for AI context
|
||||
function serializeForAi(component: ComponentModel): string {
|
||||
return JSON.stringify({
|
||||
name: component.name,
|
||||
nodes: component.nodes.map(n => ({
|
||||
type: n.type.localName,
|
||||
id: n.id,
|
||||
parameters: n.parameters,
|
||||
children: n.children?.map(c => c.id)
|
||||
})),
|
||||
connections: component.connections.map(c => ({
|
||||
from: `${c.sourceNode.id}.${c.sourcePort}`,
|
||||
to: `${c.targetNode.id}.${c.targetPort}`
|
||||
}))
|
||||
}, null, 2);
|
||||
}
|
||||
```
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
### Token Management
|
||||
- Keep prompts concise
|
||||
- Truncate large components
|
||||
- Cache common patterns locally
|
||||
- Batch similar requests
|
||||
|
||||
### Response Times
|
||||
- Scaffold generation: < 30 seconds
|
||||
- Suggestions: < 500ms (local), < 3s (AI)
|
||||
- NL parsing: < 3 seconds
|
||||
- Design analysis: < 5 seconds
|
||||
|
||||
### Offline Support
|
||||
- Local pattern library for suggestions
|
||||
- Cached design rules
|
||||
- Basic NL patterns
|
||||
- Graceful degradation
|
||||
|
||||
## Settings & Configuration
|
||||
|
||||
```typescript
|
||||
interface AiConfiguration {
|
||||
// API
|
||||
apiKey: string;
|
||||
apiEndpoint: string; // For custom/proxy
|
||||
model: string;
|
||||
|
||||
// Features
|
||||
features: {
|
||||
scaffolding: boolean;
|
||||
suggestions: boolean;
|
||||
naturalLanguage: boolean;
|
||||
designAssistance: boolean;
|
||||
};
|
||||
|
||||
// Suggestions
|
||||
suggestions: {
|
||||
enabled: boolean;
|
||||
frequency: 'always' | 'sometimes' | 'manual';
|
||||
showInline: boolean;
|
||||
showPanel: boolean;
|
||||
};
|
||||
|
||||
// Design
|
||||
design: {
|
||||
autoAnalyze: boolean;
|
||||
showInCanvas: boolean;
|
||||
strictAccessibility: boolean;
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
### Unit Tests
|
||||
- Prompt generation
|
||||
- Response parsing
|
||||
- Pattern matching
|
||||
- Change generation
|
||||
|
||||
### Integration Tests
|
||||
- Full scaffold flow
|
||||
- Suggestion pipeline
|
||||
- NL command execution
|
||||
- Design analysis
|
||||
|
||||
### Manual Testing
|
||||
- Various project descriptions
|
||||
- Edge case components
|
||||
- Complex NL commands
|
||||
- Accessibility scenarios
|
||||
|
||||
## Cline Usage Notes
|
||||
|
||||
### Before Starting Each Task
|
||||
|
||||
1. Read existing AI infrastructure:
|
||||
- `AiAssistantModel.ts`
|
||||
- Related AI components in `noodl-core-ui`
|
||||
2. Understand prompt patterns from existing templates
|
||||
3. Review how changes are applied to node graph
|
||||
|
||||
### Key Integration Points
|
||||
|
||||
1. **Node Graph**: All changes go through `NodeGraphModel`
|
||||
2. **Undo/Redo**: Must integrate with `UndoManager`
|
||||
3. **Project Model**: Scaffolds create full project structure
|
||||
4. **Settings**: Store in `EditorSettings`
|
||||
|
||||
### API Key Handling
|
||||
|
||||
- Never log API keys
|
||||
- Store securely (electron safeStorage)
|
||||
- Clear from memory after use
|
||||
- Support environment variable override
|
||||
|
||||
## Success Criteria (Series Complete)
|
||||
|
||||
1. ✅ Users can create projects from descriptions
|
||||
2. ✅ Contextual suggestions appear while building
|
||||
3. ✅ Natural language commands modify components
|
||||
4. ✅ Design issues automatically detected
|
||||
5. ✅ One-click fixes for common issues
|
||||
6. ✅ Works offline with reduced functionality
|
||||
|
||||
## Future Work (Post-AI Series)
|
||||
|
||||
The AI series enables:
|
||||
- **Voice Control**: Voice input for commands
|
||||
- **Image to Project**: Screenshot to scaffold
|
||||
- **Code Generation**: Export to React/Vue
|
||||
- **AI Debugging**: Debug logic issues
|
||||
- **Performance Optimization**: AI-suggested optimizations
|
||||
|
||||
## Files in This Series
|
||||
|
||||
- `AI-001-ai-project-scaffolding.md`
|
||||
- `AI-002-ai-component-suggestions.md`
|
||||
- `AI-003-natural-language-editing.md`
|
||||
- `AI-004-ai-design-assistance.md`
|
||||
- `AI-OVERVIEW.md` (this file)
|
||||
|
||||
## External Dependencies
|
||||
|
||||
### Anthropic API
|
||||
- Model: claude-sonnet-4-20250514 (default)
|
||||
- Rate limits: Handle gracefully
|
||||
- Costs: Optimize token usage
|
||||
|
||||
### No Additional Packages Required
|
||||
- Uses existing HTTP infrastructure
|
||||
- No additional AI libraries needed
|
||||
Reference in New Issue
Block a user