Revamp Workflow Schema: Minimal, Declarative, Smart-Engine

Are you tired of overly complicated workflow configurations? Do you find yourself wrestling with schemas that seem to demand you know every nitty-gritty detail of the system's inner workings? If so, you're not alone. We're excited to propose a significant revamp of the workflows.yaml schema, moving towards a minimal, declarative, and profile-agnostic system. Our core idea is to shift complexity away from configuration and where it truly belongs: the workflow engine. Imagine a system where authoring workflows is a breeze, evolution is stable, and the engine is smart enough to handle advanced features like MCP integration, file handling, model routing, and multi-step reasoning – all without needing constant schema updates. This proposal aims to strip away unnecessary fields and replace them with a clean, future-proof structure, powered by intelligent, engine-level detection. We believe this will dramatically simplify the user experience and set a robust foundation for future enhancements.

🎯 Motivation: Why the Change?

The original workflow schema was an ambitious attempt to explicitly define various aspects of workflow execution. It tried to dictate things like the types of input a step would accept, whether files were allowed, the specific profile each step should use, and when Jira/GitHub MCP should be activated. While well-intentioned, this approach has, over time, resulted in configurations that are undeniably verbose, create a high-friction authoring experience, and become painful to maintain. More critically, it has misaligned with the reality of how users actually interact with AI Developer. Our workflows shouldn't be burdened with understanding low-level implementation details. Questions like, "Is this a Jira ticket?", "Should files be accepted for this step?", "Which assistant or profile should handle this particular task?", or "Should we activate issue MCP for this specific operation?" are responsibilities of the engine, not the configuration. By simplifying the schema, we empower workflows to focus solely on what needs to be done, leaving the how to the intelligent workflow engine. This separation of concerns is crucial for scalability, maintainability, and a significantly improved user experience. We want to empower developers to focus on the logic and intent of their workflows, not on the plumbing required to make them run. This shift is not just about making things easier; it's about building a more robust and adaptable system for the long haul, ensuring that as our capabilities grow, our workflow definitions remain elegant and straightforward.

🧠 Core Idea: Shifting Intelligence to the Engine

The fundamental principle behind this revamp is to move all intelligence into the runtime engine. Instead of the workflow schema dictating every nuance of execution, the engine will become the orchestrator of smart behaviors. This means the engine will be responsible for smart issue detection, automatically recognizing and integrating with platforms like Jira or GitHub when relevant references are detected. It will handle smart file inclusion, always allowing files to be passed and intelligently determining their relevance within the workflow context. Smart prompt resolution will be managed through profiles, which will reside exclusively at runtime, allowing for dynamic selection. Similarly, smart tool selection will default to the profile's designated tool unless explicitly overridden in the workflow step. Finally, smart context merging will ensure that the LLM result from each step seamlessly becomes the input for the subsequent step, creating a fluid and dynamic execution flow. By centralizing these intelligent behaviors within the engine, the workflow definitions themselves can remain purely declarative and astonishingly simple. This approach not only streamlines the authoring process but also makes the entire system far more resilient to change and easier to extend with new capabilities in the future. It’s about building a system where the configuration describes intent, and the engine handles the intelligent execution of that intent.

🚀 Finalized Workflow Schema: Simplicity is Key

Let's look at the tangible difference this schema revamp makes. Before, our workflows.yaml files were cluttered with fields such as input.kind, accepts_file, profile, and uses_issue_mcp, among others. These fields, while seemingly necessary at the time, added significant overhead and complexity. After this revamp, the new schema is elegantly minimal. A typical workflow step will now look like this:

workflows:
  workflow_name:
    description: "Human-readable summary of the workflow"
    steps:
      - name: step_name
        prompt: prompt_key
        tool: optional_specific_tool
        timeout_sec: optional
        retries: optional

That’s it. Notice what’s missing? There's no input classification, no profile selection directly in the workflow, no explicit file permissions, and absolutely no MCP flags. This deliberate exclusion means that nothing leaks implementation detail into the configuration. The workflow focuses purely on the sequence of prompts and optional tool overrides. This clean structure ensures that authors don't need to concern themselves with the underlying mechanisms of MCP, profile management, or how files are handled. They can simply define the logical steps of their process. This radical simplification is a cornerstone of our strategy to make the system more accessible, maintainable, and future-proof. By removing these extraneous fields, we drastically reduce the learning curve and the potential for configuration errors, allowing users to focus on the core logic of their AI-driven processes.

🤖 Engine Behavior: The New Smart Model

With the schema simplification comes a powerful expansion of the engine's responsibilities. The engine now becomes the central hub for intelligent behavior, orchestrating complex actions that were previously scattered across configurations. Firstly, Smart Issue Detection will allow the engine to automatically recognize Jira (ABC-1234) or GitHub (https://github.com/.../issues/42) references, or even enterprise Atlassian links (https://yourcompany.atlassian.net/...), and hydrate the workflow context using MCP. Secondly, Smart File Handling means files are always permitted; the engine will intelligently determine their relevance to a given step, eliminating the need for explicit accepts_file flags. Thirdly, Profile-Agnostic Workflows are enabled because profiles will now exist exclusively at runtime. They can be selected via CLI arguments (--profile), environment variables, or fall back to a default. Profiles themselves will encapsulate crucial configurations like the system prompt, the chosen model, prompt templates, and default tools, abstracting these details away from the workflow definition. Fourthly, Context Merging ensures a smooth flow between steps. Each step will receive a context object ctx containing text, files, and crucially, last_result from the previous step, enabling iterative processing. Lastly, Smart Tool Selection offers flexibility: a step can optionally specify a tool (e.g., tool: aider), but if omitted, the engine will default to using the tool defined in the active runtime profile. This redefined engine behavior transforms it into a highly capable, context-aware orchestrator, simplifying workflows and enhancing their power.

💎 Value: Why This Matters

This strategic shift towards a minimal, declarative schema and a smarter engine offers a wealth of benefits. Firstly, it dramatically simplifies the authoring of workflows. Anyone, regardless of their familiarity with the intricate details of MCP, profiles, or advanced LLM routing, can now write effective workflows. They simply describe what needs to happen. Secondly, it drastically reduces the need for schema changes. As our system evolves and new features are added – like conditional execution, branching, or support for new MCP platforms – the core schema will remain stable, preventing the maintenance headaches associated with frequent schema updates. Thirdly, the system becomes future-proof and far more extensible. This engine-level smart behavior lays the essential groundwork for advanced capabilities such as conditional execution, branching workflows, Directed Acyclic Graph (DAG) workflows, sophisticated multi-agent tool routing, and even the auto-selection of various MCPs (Slack, GitLab, Linear, etc.) or repository-aware contextual workflows. Fourthly, it establishes a cleaner mental model: workflows clearly define what happens, and the engine intelligently determines how it happens. This separation of concerns is invaluable for understanding and debugging. Fifthly, it minimizes configuration drift. Gone are the days of misplaced or incorrect flags like uses_issue_mcp: true or conflicting step-level profile definitions, leading to more predictable and reliable execution. Finally, it significantly eases debugging and testing. By centralizing complex logic within the engine, testing and verifying behavior becomes a more contained and manageable process, rather than being spread across numerous configuration files. This holistic approach ensures a more robust, adaptable, and user-friendly system.

📌 Action Items: Making It Happen

To successfully implement this vision, we need to tackle several key areas. Our immediate priority is to simplify the schema itself. This involves removing deprecated fields like profile, input.kind, accepts_file, and uses_issue_mcp from both the loader logic and the schema definitions, fully adopting the new, minimal structure. Concurrently, we must update the workflow loader to parse only the essential description and steps[] components. The most substantial effort lies in implementing the engine's smart detection capabilities. This includes developing robust logic for Jira/GitHub issue detection, intelligent file inclusion, prompt-based issue-context detection, and seamless step context merging. As a recommended, though optional, enhancement, we should update the prompt registry to allow for metadata, such as requires_issue_context, which can provide valuable hints to the engine. Once the core engine and schema changes are in place, we need to systematically rewrite all existing workflows, meticulously removing deprecated fields and reformatting them according to the new schema. Finally, a crucial step for user adoption and understanding is to update our documentation. This should include a new section detailing "Workflow Schema v2", providing clear engine overview diagrams, and offering practical examples of both simple and advanced workflows that leverage the new architecture. Completing these action items will ensure a smooth transition and unlock the full potential of the revamped system.

🏁 Outcome: A Streamlined Future

By embracing this proposal, we are set to deliver a streamlined, powerful, and significantly more flexible workflow system. The resulting architecture will be easier to use, dramatically easier to maintain, and fundamentally more robust. It aligns perfectly with the intended operational model of AI Developer, where the focus is on intelligent automation guided by clear intent. This revamp is not merely an incremental improvement; it’s a foundational shift that positions us for long-term evolution. It lays the critical groundwork for advanced features like multi-agent collaborations, DAG-based workflows, and dynamic integrations, all without the specter of redesigning the schema again. We are building a system that can grow and adapt gracefully, ensuring that as AI Developer's capabilities expand, its workflow management remains elegant, efficient, and ahead of the curve. This is about creating a sustainable and scalable platform for the future of AI-powered development workflows.

For further reading on workflow management and declarative systems, you might find insights from The Apache Airflow documentation on their approach to defining complex data pipelines to be valuable. Additionally, exploring Kubernetes' declarative configuration principles can offer a different perspective on managing complex systems through desired states.