Research

Wix is a popular website builder that allows users to build all kinds of websites, ranging from personal blogs, company websites to storefronts. Website creation is done entirely through the editing studio, without the user needing to write code at all. Wix extends the website creation concept further by allowing users to host their websites on Wix’s domain, and review analytics such as viewer traffic. In 2016, Wix embarked on implementing AI for website design [1] and currently features AI tools that aid in section creation and text generation.

Feature	Our Evaluation	Include?
No code needed for designing	Allows for users to focus primarily on design and not worry about coding.	Yes
Allow hosting websites on domain	Wix does not offer an exporting to code function, rather users host their websites on their domain. With the exporting feature of Blueprint AI into various GUI frameworks, this feature is counterintuitive as users would export UI code to integrate with their own backend for greater control over publishing their product.	No
AI tools, multiple options generation	Wix's AI tools allow for text generation and also section creation. This is highly versatile and useful. The AI also generates multiple design options for the user to choose from which is also very practical for the user.	Yes

Microsoft PowerPages is a platform designed for rapid website development, offering a user-friendly drag-and-drop interface builder that enables users to create applications with ease. The platform leverages AI-assisted UI recommendations to enhance the design process. PowerPages seamlessly integrates with a variety of backend services such as its Dataverse storage capabilities, security authentication features and analytics functionalities [2]. One key feature of PowerPages that set it apart from more mainstream website creators is its ability to export the code in HTML, CSS and JavaScript, the user can then further edit the code [3].

Feature	Our Evaluation	Include?
Backend services integration, security authentication	A useful feature that helps translate design into development and link front end to backend more easily. Takes the frontend development one step further. These features are locked behind a paywall and are subscription based - limiting the types of users who are willing and able to afford to use them. Blueprint AI will be a free to use product for all users to use, with no tiered features	Partial inclusion in the form of exporting the code into GUI frameworks that allow for easier integration with backend.
Exporting of code.	Exporting is a standout feature that is important for website development. As for PowerPages, this feature is limited to exporting in HTML, CSS and JavaScript and is not integrated into an IDE, changes made on the IDE are not displayed on the PowerPages app and vice versa. This adds a layer of inconvenience that Blueprint AI aims to solve.	Yes. Aim to further increase exporting capabilities and VSCode integration for ease of use.

Figma is a powerful web-based design tool known for its collaborative design interface. While Figma itself is not specifically tailored for frontend development like Blueprint AI, it offers robust UI design capabilities and can integrate with code-generation plugins.

Feature	Our Evaluation	Include?
Collaborative Design Interface	Figma allows multiple users to design simultaneously, enhancing team collaboration. This is a standout feature for UI/UX teams.	Partial Inclusion — Potential for team collaboration features but simplified for Blueprint AI.
Prototyping and Animation	Figma allows for interactive prototypes and animated transitions directly in the tool.	No — This is better suited for design rather than code generation.
Code Export through Plugins	Figma supports third-party plugins that export designs into React, Flutter, or HTML/CSS code.	Yes — A similar feature for streamlined code export will be integrated in Blueprint AI.
Auto Layout	Auto Layout enables flexible design structures that adapt to content changes, improving design consistency.	Yes — This feature aligns well with Blueprint AI’s goal to generate adaptable UI designs.
AI-Powered Features	While Figma’s AI capabilities are still evolving, some plugins offer AI-assisted content generation.	Partial Inclusion — Blueprint AI’s AI will focus on complete frontend generation, including layout and design refinement.

Visily is highly similar to other website builders like Wix but has much more capabilities and functionalities. Besides webpages, Visily can also create mobile phone pages, wireframes and mockups. The use of AI is also more prominent and wider in Visily as compared to Wix as Visily allows the user to upload screenshots of existing UI to kickstart the generation process, a welcomed feature from the usual description/ text based prompts that Wix offers. Visily supports multi-page generation and a multi-page view as shown below which gives a more holistic view of the user’s design.

Feature	Our Evaluation	Include?
Can create mobile pages	As there are more mobile phones than computers on this planet [4], the need for tailored mobile page front end design is apparent. Blueprint AI will thus need to include this feature so as to increase its useability.	Yes
Use screenshots/ images as AI reference point	Screenshots are a quick and fantastic way to tell the AI exactly what you want generated as describing with text prompts can get too tedious and is also difficult to describe clearly to the AI.	Yes
Multi-page generation and editing	Frontend development involves more than just one page, having AI and a studio that can view, edit and create multiple pages would be extremely useful in creating a cohesive and consistent frontend design for the user.	Yes

Our first project brief stated that we were to improve upon a project made by a group of UCL postgraduate students in 2024 for their IXN module titled Microsoft Copilot UI v1.0. The scope of this was largely similar to Blueprint AI, with the main feature of incorporating AI to generate, edit and export user interfaces all within a VSCode extension. Our initial brief wanted us to explore, using the existing codebase, other frameworks for exporting the frontend designs into frontend frameworks.

The link to the entire repository can be found in [5], including a demo video of that project.

Overview of Microsoft Copilot UI v1.0

Copilot UI v1.0 is a webview-based React editor integrated into a VSCode Extension, primarily written in Typescript. The main backend code, consisting of the main program logic (API calls, testing, JSON tree conversion) is located in the src directory while the frontend of v1.0 is found in the webview-ui directory. The system architecture of v1.0 is shown in Figure 1, this is taken directly from the GitHub repository. The key features of the project and brief explanations are listed in the table below.

Figure 1. Copilot UI v1.0 System Architecture

Key Feature	Explanation
VSCode Extension Wrapper File reference: .vscode	VSCode API interacts with the editor [6], workspace, and key storage. Entry point for users to interact with the system, such as opening the webview, saving/loading projects, and triggering AI generation.
React Editor	Craft.js was used as a framework for building drag-and-drop page editors. FluentUI, Microsoft’s design system, was used for the editor's components like text boxes. React Grid Layout was used to manage the layout of components within the editor, providing responsive and resizable grids.
AI integration File references: src/generateImage src/generateLayout	Azure, OpenAI GPT-4: Used for generating text layouts and processing natural language inputs such as the text prompts. DALL-E 3: Used for generating images based on user prompts or layout schemas like screenshots of websites. Users input and use their own API keys.
Generated Layout Node Tree File reference: src/generateLayout/buildNodeTree.ts	The system uses AI (e.g., GPT-4) to interpret the input and generate a corresponding layout node tree. JSON-based tree structure representing the arrangement and properties of components in a design is generated and displayed in the editor.
Exporting Frameworks File references: src/HTML src/WinUI	HTML + CSS: Converts the JSON node tree into HTML and CSS for web use. WinUI: Converts the design into WinUI format compatible with Windows applications [7].

Learning points

1. Good system architecture

The system architecture (Figure 1) is well-thought out and planned suitably to meet the requirements of the project. This gave us some semblance of structuring and organising such a complicated system. The VSCode extension structure can be replicated and modeled for our version of the project as it relies on the existing, well-documented VSCode API library [6]. Figure 2 shows a code snippet taken from the file in: src/generateLayout/generateLayout.ts

This file, along with other similar files found in the generateLayout directory was useful as it allowed us to adapt our project based on this similar structure of importing the AI API library and component types, making API calls to OpenAI. Furthermore, we can also improve on the meta prompts (Figure 2, line 10) beyond just safety and ethics, to use them to refine the generated node tree. Meta prompts are an excellent way of guiding large language models to perform complex tasks and reasoning in a more methodical and logical way [8]. Based on the research paper in [8], we can leverage the use of meta prompts to generate more refined and tailored results for the user.

Figure 2. Code snippet of generatedLayout.ts taken from v1.0

2. Choice of Frameworks

The implementation of widely-used, industry relevant frameworks such as React and WinUI helped us to scope our research more clearly. This gave us a good sense of what we could start exploring and looking into without having to write our own components from scratch. This is discussed further in section 2: Technology Review.

This project was a useful example of the existing frameworks we can use, where v1.0 highlights the ability to successfully implement such frameworks into our version - with the method of exporting to HTML + CSS and WinUI frameworks being evident with v1.0.

Conclusion

Version 1.0 gave us a good starting point and provided enough direction and structure to aid us in our research and development process. It gave us confidence that programming a system like this is feasible. In particular, the frameworks and design decisions made gave us a reference point and insight on how it could be implemented, especially the exporting of UI designs into various frontend frameworks and also structuring our output using data structures like node trees.

Pain points

1. Complex Directory Structure

While the system architecture design provided a clear high level overview of the system, the directory structure of v1.0 is complex and difficult to navigate. Several issues stood out which made understanding the system more challenging.

There were multiple nested folders containing different functionalities, causing a bloated folder hierarchy. For instance, the tests (__tests__) were located in the backed folder (/src) and this folder included more folders containing different types of tests. A modular approach can be employed for our version, to reduce folder sprawl and improve organisation [9]; thereby enabling our system to be scalable and extendable.

Additionally, several files included were obsolete and design decisions that were not made were left in the source code. This made the directory structure unnecessarily convoluted. It is evident that the group attempted to integrate some form of GitHub Copilot calls in the backend but was ultimately abandoned. While there was some explanation of what this feature could potentially be, the implementation was partially done with boilerplate code (Figure 3). As such, this code served no purpose but to make the source code files more complicated.

Figure 3. Abandoned code for integrating Copilot left in source code

2. Nested components and mixed UI logic

Some crucial components of the code were written with nested components, incorporating mixed UI logic in the program. This poor programming practice made deciphering the program logic more tedious and also harder to maintain, scale and test. One main culprit is the MainWebviewPanel.ts program that manages a webview panel in the VSCode extension. This class is responsible for creating, displaying, and handling interactions with the webview panel, which is a vital component of the system - yet landed itself to the aforementioned faults.

The _setWebviewMessageListener method in MainWebviewPanel.ts contains a large switch statement with deeply nested logic for handling various commands (Figure 4). Each case in the switch statement performs complex operations, such as: Fetching API keys, Saving and loading files, Processing sketches and text descriptions, etc. This, among other issues, makes the method too long and complex, which reduces r eadability and maintainability.

Figure 4. Snippet of _setWebviewMesageListener method containing many switch statements

Verdict

While v1.0 provided a sufficient basis to scope our research and structure our project, some major issues such as the complex directory structure and complicated code logic made it difficult to extend v1.0. As such, we made the difficult but necessary decision to redo the system from scratch in our own way. Though this may seem like a poor decision in terms of time, redoing the system allowed us to overcome the issues v1.0 faced with the fundamental design of the system, thereby allowing us greater flexibility in implementing better, more robust features. With a more modular approach and design for our version, we can ensure greater maintainability, reusability, scalability and abstraction. A version that emphasizes the SOLID principles [10] will greatly improve system architecture, allowing for future COMP0016 groups to extend our project with more ease than it took us to understand v1.0’s system.

For solutions to a system like Blueprint AI, we considered two aspects.

1. Presence of Coding

This aspect refers to how much the user will need to code to get their frontend designs out and working, also taking into account a user’s programming skill level. The increasing prevalence of low-code/no-code products such as Wix and Microsoft PowerApps signals a shift in traditional software development workflow, reducing the barriers to entry for software development. This is greatly empowering for the average user as the technical skills needed are vastly reduced [11]. Furthermore, the estimated low-code/no-code market size was USD 6.78 billion in 2022, and is projected to grow significantly in the next decade [12].

While a low-code/no-code design choice would make sense given the current market, this would limit the flexibility and control a user has when designing their frontend. The user would be limited to pre-designed components and structures, which is restrictive [13]. Additionally, employing a low-code/no-code design would not make our student-developed project standout against industry standard solutions like Wix and PowerApps which are already established.

On the other hand, incorporating too many elements of coding required by the user would cater towards the more experienced developers and would be more time-consuming; a counter-intuitive design choice for Blueprint AI.

With the intended users being those who require a quick, reliable way to scaffold and generate frontend designs - regardless of their know-how and technical expertise in frontend development, our approach would need to cater to this type of user. As such, an approach requiring more coding than conventional low-code/no-code platforms but at the same time not be a necessary component for the frontend generation. This comes in the form of the exporting feature where users can edit the exported code in the frontend framework they need, a welcomed feature over low-code/no-code platforms like Wix which lack an exporting feature. Users of all programming backgrounds would thus find that using Blueprint AI is valuable in terms of the time savings it affords and also the flexibility in editing the frontend code generated. Figure 5 depicts a general indication of where Blueprint AI falls in the spectrum of coding required by the user.

Figure 5. Coding spectrum approach used for Blueprint AI

2. Prominence of AI

Currently, popular industry solutions like Wix, Visily and PowerApps do feature AI tools to help generate frontend designs (as discussed under Section 1.1), but these AI tools are relegated to being an additional feature rather than being integrated throughout every aspect that the user interacts with when designing. We aim to get Blueprint AI to stand out from existing solutions through making AI more prominent; studies have shown that AI is extremely useful in expediting frontend development, thereby alleviating workloads through automation [14].

In order to make AI more prominent, from the start, users are prompted to tell the AI what they want through text, image or both. We also plan to include a chat bot for the user to interact with the AI by telling it what changes to make. This allows for prompts by the user that can be tailored to the users specific needs, enhancing the flexibility and control that the user has as well - beyond manually tweaking the frontend source code themselves. Additionally, the AI also recommends different pages that the users need as frontend design requires multiple pages, on average 12-30 pages for a full-fledged website - according to [15].

With this, at every step, AI is integrated seamlessly into the frontend development experience, streamlining the development process. With AI-driven UI generation, users are afforded efficiency and flexibility. Figure 6 depicts the continuous integration of AI in Blueprint AI.

Figure 6. Continuous integration of AI in Blueprint AI

One of the standout features of v1.0 was its integration as a VSCode extension. The sheer user base VSCode has, amounting to over 2.5 million active monthly users in 2017 [16] makes it an attractive choice to deploy Blueprint AI as a VSCode extension. The free-to-use IDE hosts over 4000 extensions [16] and has comprehensive documentation on how to deploy our project as an extension [6]. Thus, having Blueprint AI as a VSCode extension means it runs on any device that supports VSCode, removing hardware dependency concerns and allowing our project to reach a wide range of developers.

For UI layout recognition, we needed software that could effectively and efficiently extract text description from given screenshots of websites. This text description would then be used to describe to the AI what kind of website the user wants to build. With this as a priority, we considered the following approaches:

1. Traditional Image Processing (OpenCV, cv2)

OpenCV (Open Source Computer Vision Library) is a computer vision and machine learning software library. It is widely used for real-time image and video processing, and it provides a comprehensive set of tools for tasks like object detection, facial recognition, motion tracking, and more. Its cross platform capabilities and common programming language interfaces (C, C++, Python, Java) make it easier to develop with [17].

Pros	Cons
No licensing cost, Extensive Documentation, Versatile as it supports a wide range of computer vision tasks.	Steep learning curve owing to wide range of advanced tools available

For Blueprint AI, there is simply no need for real-time image processing. This allowed us to narrow the search for libraries that related to text description extraction from an image, overcoming the need to pore through advanced libraries for heavy-duty image processing. Given that OpenCV is also widely used in industry, it made OpenCV a suitable choice.

Deep learning based Optical Character Recognition (OCR):

OCR is a technology that converts images (e.g., scanned documents, photos, or screenshots) into text that computers can read and edit . While OCRs can defer in the types of images they can extract from, the main function of an OCR is to allow computers to recognise and extract text from images. Some OCRs leverage AI and deep learning in the image processing and text detection and recognition front, allowing for more robust and impressive functionality [18]. This use of AI by OCR is inline with Blueprint AI’s central theme and abstract, as such, we researched on OCRs that specialised in extracting from screenshots that have deep learning capabilities.

We narrowed our options down to two competing solutions: EasyOCR and Tesseract. Both are open-source systems which use AI for processing. The table below shows what each OCR can do.

OCR	Description	Standout Features	Drawbacks
EasyOCR	Python based OCR which has support for over 80 languages. Built on Pytorch and easy to implement and use [19]. Faster on GPU due to its deep learning-based architecture.	Provides higher accuracy for noisy, complicated images. This can be useful for complicated website layouts such as Chinese e-commerce platforms, which are known to be more intricate compared to Western counterparts [20]. Handles multilingual text in a single image effectively	Limited fine-tuning options.
Tesseract	PA traditional OCR engine developed in part by Google with comprehensive documentation found in [21]. Optimized for CPU and generally faster on CPU.	Highly customisable, supports custom training and fine tuning. Affords high accuracy with clean input.	Struggles with noisy or distorted text. Requires separate language data files for each language.

We decided to use EasyOCR, with the primary reason being the ease of installation and implementation with Python and its simple API. Tesseract would require more set-up to get it running and implementation would be a bit more tedious. Additionally, while both OCRs would need some form of image pre-processing to be done before character recognition, EasyOCR requires less pre-processing. Lastly, while AI capabilities are an important factor for Blueprint AI, fine-tuning the OCR to produce the highest quality result is not a priority as we want our system to focus more on efficiency and speed for our users to rapidly scaffold their frontend development.

In Blueprint AI, we use cv2 (OpenCV) to clean up images before running them through EasyOCR for text extraction. Think of cv2 as a tool that prepares the image—adjusting brightness, removing noise, and sharpening text—so it’s easier to read. This step is important because raw screenshots can be cluttered or blurry, making it harder for the OCR to recognize text accurately. Once the image is prepped, EasyOCR, a deep learning-based tool, quickly scans and extracts the text, even from complex website layouts. This setup helps us efficiently turn website screenshots into clear, structured descriptions that the AI can use to generate front-end designs.

1. Programming Languages and Frameworks

We decided to continue the use of React, CraftJS and FluentUI from v1.0. React’s component-based architecture allows us to break down UI components into smaller, reusable pieces. This modularity is perfect for generating dynamic UIs, as components can be composed and reused across the application, ensuring ease of generating UIs dynamically based on user and AI input.

CraftJS provides a drag-and-drop interface for building and editing UIs. This makes it easier for non-technical users or designers to create and modify layouts without writing code. Also, CraftJS integrates with React components which makes the system more cohesive.

With these frameworks, we used Typescript instead of Javascript because the static typing of Typescript allowed for catching errors at compile time rather than runtime, easing the development and debugging process significantly. This also helps ensure the project remains scalable and maintainable as it grows as Typescript is better for organisation and refactoring.

2. AI API

For Blueprint AI’s backend, we leverage LangChain and Axios to streamline AI interactions and data fetching. LangChain is a powerful framework designed to work with large language models (LLMs), making it easier to build AI-driven applications by managing prompt chaining, memory, and model selection. This allows Blueprint AI to create more dynamic and context-aware interactions when generating front-end designs. Meanwhile, Axios, a popular JavaScript library for making HTTP requests, ensures smooth and efficient communication with APIs, including OpenAI’s models. Its promise-based structure simplifies data handling, making API calls more reliable and easier to manage. Together, LangChain and Axios help optimize AI integration, ensuring a seamless and responsive user experience.

We have also decided to stick with using OpenAI for the AI API. OpenAI contains many different models suitable for this project, such as DALLE, a generative AI model that creates images from text descriptions, allowing for image generation. OpenAI also boasts industry leading models that are highly advanced and intelligent, outperforming other competitors like Meta, Amazon and Microsoft [22]. With the impressive line-up of AI models, users also have more flexibility in managing their own tokens to manage their costs. Combined with the fact that OpenAI is also highly popular and widely used [23], and integrating OpenAI API calls into the backend is relatively straightforward, Blueprint AI will use OpenAI for its AI integration.