How AI and humans work together to build your app

What: The brain that powers our platform
At Builder.ai, we think of our Builder Knowledge Graph as the brain of our operations, because it powers most of our workflows. Its primary objective is to fill in the gaps between what makes your idea special and what it’ll need to be special.

The graph contains the entities in our Builder.ai ecosystem (that’s everything from features, to projects and developers). And it shows how all these are connected. For example, a developer might be working on a specific feature that’s part of a bigger project.

Then we use graph embedding algorithms: they translate this information, so that it can be read and used by machine learning algorithms. This allows us to predict time and costs for new projects. It also lets us split your project into parallel work streams and find the best developers to work on each task.

Tech used: graph paradigm’s flexibility lets us calculate metrics like feature node centrality and degree. Neighbourhood detection for recommendation systems and feature link prediction – generates our instant prototypes. Knowledge Graph generates subgraph and node embeddings – a powerful tool for numerous machine learning models, using MetaPath2Vec and Node2Vec.

What: Our intelligent conversational assistant system
She’s integrated into the Builder.ai platform (Studio Pro), Builder 360 and Natasha Cockpit. Natasha uses a proprietary dialogue manager and the Builder Knowledge Graph to guide you as you concept and design your idea.

How is Natasha able to offer you suggestions? We use an arsenal of data and machine learning algorithms to support feature and template recommendations, negative feature recommendations and feature tagging.

Our mission? For Natasha to one day become the voice of Builder.ai to provide true AI-assisted, self service on our platforms.

Tech used: cutting edge natural language understanding, like Flair for Named Entity Recognition and neural based sentence embedding models. Graph embedding models such as Metapath2Vec and GraphSage.

What: Natasha improving your spec call experience
We’re using Natasha Cockpit to improve your experience on that all-important spec call with us – where you describe what you want and we create a blueprint for your new app.

Eventually, each conversation will be powered by Natasha, providing our teams with the right question to ask you. She then listens for your answer and adds the relevant features to your Buildcard.

This will allow us to be more precise and ensure that every person you speak to at Builder.ai is an expert in what you’re trying to build. Natasha Cockpit is powered by our speech and conversational analysis system.

Tech used: Deep learning for NLP, unsupervised learning. Transformers are applied to understand similarity between phrases. Hierarchical clustering then uncovers topics in chat.

What: Code components that we reuse across multiple Buildcards
Building Blocks reduce both your costs and timeline, while maintaining standard code quality across all our Buildcards.

Each Building Block has two sections – core & custom:

Eventually, each conversation will be powered by Natasha, providing our teams with the right question to ask you. She then listens for your answer and adds the relevant features to your Buildcard.

Core sections: the lines of code which represent the main functionality and reusable components across Buildcards. (We use MD5 hashing technology to maintain the reusability of every block.)

Custom sections: the snippets of code that define customisations that are specific to your Buildcard. This can include placeholder text, theme, colour, font, error messages or branding information.

What: The platform that assembles our Building Blocks
Code Engine is a platform which assembles all the Building Blocks needed for your Buildcard (based on the features that you’ve chosen).

Tech used: domain specific translators generate deployable code. Continuous upgrades to a Buildcard are supported by incremental runs updating the code platform with the newly added features.

What: How our Building Blocks communicate
Run Engine is the messaging channel that we use to maintain communication between our different Building Blocks within your project, or outside of it. (This includes sending HTTP/S GET or POST requests from one page to another.)

What: Design components that we reuse across many Buildcards
Each feature in our catalogue has its own design starter kit – these pre-created, design files save designers having to start from scratch. Not only does this save time and money, it also maintains design quality.

Reusing existing files lets your designer focus on customisations to suit your specific idea (without having to design the basic use cases of that feature). We’ve got design files to suit every category, this makes design discussions less subjective and speeds up concepting your app’s design.

What: Stitches your code together to create your app
Our Merge Engine automatically merges your app’s design code with its functional code. This tool also handles the navigations, which connect the different screens in your app – as well as any animations or other interactions you want on each screen.

Styles and assets are all consolidated in one place. So it’s simple for our experts to customise and use the code it’s generated for your app.

What: Automatically converts designs to code
UI Engine is a Builder.ai homegrown design-to-code product that converts design files (in Sketch, soon in Figma too) into React Code (which supports both mobile and web). It scales the generated code for different screen sizes – no effort is needed from a designer or developer.
Design files uploaded to our Designer Surface are automatically converted into corresponding web or mobile code. This system is designed in such a way that changing formats would be relatively easy.
Tech used: converts the design files into our proprietary domain specific language using adaptors – we then run translators to get the actual React/React Native code.

What: Complete partner management system
Building Blocks speed things up but as your app is unique, we need people to customise them so your app works – and looks – exactly the way you want it to. Our partners supply the designers, developers and QA experts who make this happen.

We use Builder Hive to onboard our capacity partners and test their experts. (Before we hire any third-party experts to work on your app, our in-house experts assess them.)

Builder Hive allows partners across the world to supply us with talent – while we make sure everyone’s apps are the same, consistently high standard.

Builder Hive helps us:

• Onboard partners: profile and contract completion
• Onboard experts: profiling, quiz, assessments, course completion
• Review assessments: auto review system validates the assessment (basis code, repo status, code analyser)
• Manage expert availability: schedule management and rate cards
• Allocate tasks to experts
  ‍

What: Task management tool.
Tracks and manages every task performed by the experts from our capacity partner network (as well as our internal product and engineering teams).

Once a project starts, Builder Tracker automatically adds all the common tasks required for each feature from our catalogue – based on inputs from our Builder Knowledge Graph.

This library of tasks and user stories is continuously refreshed, as new stories are created or edited. Our delivery team can manually edit these tasks based on what you’ve asked for. BRAT then allocates each task to the best expert who’s available to work on it.

What: Online space where app designs are created – and converted into code
Designer Surface is used by designers from our capacity network. It lists their assigned tasks – they upload designs as Sketch files (soon, we’ll accept Figma too) and it creates a flat file to send to you to approve. The designer makes any amends. Once approved, we’re able to extract UI code directly – turning designs into code in less than a minute.

Using our design library of precreated designs, and Builder Now, Designer Surface can immediately create a user flow of actual designs – ready for experts from our network to customise for you.

What: A virtual desktop that gives developers working for us, the tools they need
Our Developer Surface is where developers from our capacity partner network do the tasks they’ve been assigned. It’s a virtual desktop with all the tools and packages they’ll need to develop your app, already installed.

It uses facial recognition to make sure the named expert is the person working on the code strands. It also captures all keystrokes to ensure we know, at an industrial scale, if there’s any code being copied in, and where it’s being copied from.

Experts can access Developer Surface from a web browser, on any device, offering flexibility for them to work in a secured environment, from anywhere.

Tech used: Facial recognition. We provide minimum latency even across geographies by supporting multi-cloud regional clusters, dynamically.

What: How we test the quality of our network of experts
Our expert evaluation and grading platform allows us to continuously evaluate everyone working on your code – by giving them a developer score. These scores are integrated into our overall workflow. This helps us pick the right level of skill for each task on every Buildcard, without wasting money by giving simpler tasks to over-qualified experts.

Tech used: One of the key components is our code quality system (powered by SonarQube) to determine the score for each expert. These metrics are captured across multiple front and back end skills and fed into BRAT (our system for dynamic resource allocation).

What: Finds the best person for each task on every Buildcard
Our Better Resource Allocation Tool (or BRAT to us) generates the optimal developer assignments. Not only finding the right person for each task but splitting work into parallel swimlanes for people to work on simultaneously. This minimises cost and time.
It’s based on an operation research model and considers a plethora of information to make these assignments, including:

• Feature complexity
• Developer expertise
• Past developer experience
• Timezone
• Project affinity
  ‍

Tech used: BRAT can also quantify developer performance and expertise. It uses a live grading system that employs qualitative and quantitative feedback, based on code quality metrics.

What: Automated Quality Assurance – to check app designs
This makes sure designers’ and developers’ work on your app matches up. Visual QA compares design files with actual generated screens – and spots visual differences between the two.

It’s a pixel-to-pixel comparison, using computer vision to spot discrepancies on the static page layout of your screen (missing/spurious objects and incorrect position or colours for elements on your screens).

Visual QA logs any differences as a bug in Builder Tracker, our task management tool. Our team review it and assign it to a developer to redo.

Sometimes differences aren’t bugs (maybe they’re dynamic content or placeholder details in a form). To avoid these false positives, Visual QA masks out text elements before comparing screens.
Tech used: Computer vision. Optical Character Recognition (OCR) engine to detect and diagnose text position and spacing.
Additional routines based on optical character recognition remove text elements (see image below), before applying pixel-based diagnostics. At this latter stage, an approach based on structural similarity indices for computer vision checks element position, detects missing/spurious objects in the UI and identifies incorrect colours.

What: AI-powered software assembly line – that helps anyone build apps.
We empower anyone to turn their idea into tailor-made software, without writing any code or speaking to a developer or agency. The Studio Pro platform takes your specifications (the features your app needs and platforms it should run on) and turns them into a Buildcard (a machine readable spec for that project).

Using deep and graph learning, and bayesian estimation, it calculates the cost and approximate timeline for your chosen set of features. It’s so powerful that we can guarantee upfront that you won’t pay more than this price for this combination of features.

It also uses our Knowledge Graph to analyse the past history of feature combinations to detect where potential conflict points may emerge and factors that into its cost and approximate timeline.

Natasha, our AI conversational agent, guides you through this process and can recommend templates and features based on your replies using natural language processing.

What: Pre-packaged apps for small businesses.
Studio Store is a flavour of Builder Studio that automates the entire creation of an app, without involving our network of experts. We fuse together the common blocks with pre-written customization for different types of small business.
The power in this approach is that you can later upgrade and become a Studio Pro customer by simply adding a feature.

What: Creates industrial-grade applications – faster than any other platform.
Studio Rapid is a next generation no-code/low-code platform that allows enterprises to build applications for themselves. All you need is basic design knowledge and anyone from your team can build enterprise-grade apps using its intuitive interface. And you’ll do it faster than with any other platform.
In less than a day, you can have a complete native app on your phone with the Studio Rapid previewer app. This is an app that lets you instantly access all of the fully-functional apps you’ve designed with Studio Rapid and test changes in real-time.

What: App prototyping tool – click through your app and share with others
Builder Now automatically creates an interactive prototype for the features you’ve selected in your Buildcard. It calculates which features to link to others in a logical manner and stitches that flow into a prototype.

To do that it uses information from the Builder Knowledge Graph (it’s based on app templates, previous client projects, prototypes and our own content management tool). Then your prototype is run through a machine learning model and finally, we built an algorithm to make sure your users can get to every feature from your start screen.

You can edit it to improve user journeys and the flow of the app. The free tool creates a prototype instantly. The paid version gives you more control, customisation and designed screens, in a few days.

Tech used: MetaPath2Vec embeddings generated from the Builder Knowledge Graph are passed to a Gradient Boosting model that predicts linkage between any 2 features. Then a secondary model selects the most likely starting screen for each prototype. As these 2 models can’t guarantee a path from the starting screen to every feature in the Buildcard, we invested in our own proprietary pathfinding algorithm. It uses embeddings and an adjacency matrix to guarantee these paths.

What: Dashboard that lets you view your Buildcard’s progress.
This dashboard shows live progress as a percentage, the status of each feature and the latest release. Stores documents (invoices, contracts, receipts…). Raise tickets for bugs, queries or to-dos – and get them resolved quickly. You can ask questions and book meetings with our team here too.

Both you and our Productologists can view and resolve all dependencies here. It’s designed to reduce offline communication and offer transparency to everyone.

What: The central nervous system of the Studio platform.
Our delivery team uses Builder 360 to manage their Buildcards from payment to handover.
Builder 360 also stores and manages all demo and spec calls (from making sure follow up calls are with the same person, to allowing teams to check recordings before sending the final Buildcard to you).
‍
How Builder 360 helps us

• Displays Buildcard information – your requirements, information and selections
• Shows all statuses on all Buildcards (from unpaid to complete)
• Empowers delivery to manage their assigned customers – through toolkits in each Buildcard (chat, email, hosting, third-party integrations)
• Tracks progress – it integrates with Builder Tracker, UIW, Builder Hive
• Integrates with assembly line – for code repository creation, infrastructure creation and management, expert and delivery access/management
• Manages customers – meetings, documentation, dependencies (To-dos/queries), payments, CSAT, release management
• Manages teams – squads (Productologists, Ninjas and Senseis)
• Books and manages calls – both spec and demo calls
• Syncs meetings with Zoom, Builder Meet and Google Calendar
  ‍

What: Predicts which cloud services each Buildcard needs
Gives you an early estimate of your cloud costs at the Buildcard stage, to help you decide if you want to opt in or out of Builder Cloud.

Our algorithm analyses each feature you’ve chosen and looks at the different providers to find the best cloud provider for your app, at the lowest cost.

Choose Builder Cloud and this algorithm then automates your whole cloud provisioning cycle. That’s everything from creating cloud accounts, to setting up CI/CD, (continuous integration/continuous delivery is an automated way to integrate new code into your app), and scaling your business.

Tech used: converts data from multiple cloud providers into a domain specific language, to predict infrastructure blueprints in a cloud agnostic manner. Sets up CI/CD for automated deployments.

What: Algorithm that allows us to offer cloud savings to customers
We’ve developed an analysis tool that automates and manages our cloud reservations, called Osiris. It makes sure we always have enough cloud for our customers, without overspending and buying more than we need.

• Runs in the backend and constantly analyses our cloud reservations, based on demand.
• Analyses 5TB of usage data every month and stores that in a data warehouse.
• Uses advanced data models to predict the usage quantity and trends.
• Manages inventory and purchases automatically, without any human involvement.
  ‍

Tech used: a combination of ETL pipeline, which parses 5TB of usage data a month. Advanced data models such as SARIMAX predict usage quantity and trends. Modified version of knapsack algorithm recommends the commitments.

What: The data backbone of our applications
We use our Catalogue Management Tool to allow our different services to communicate with each other, in one place. Our Catalogue Management Tool also creates Gitlab repositories for projects (this is where you can store code and make changes to it).

What: Manages all your third-party services
Our platform provisioning service brings together – in a single place – all the third-party tools and services you might need to build, run and scale your software, with us.

It aggregates software services: from payment gateways and logistics, to domains and licences (like Office 365), and automatically provisions accounts for these services with a click of a button. So you avoid the hassle of juggling between multiple platforms.

Choose services on demand you’d like to integrate with your app and we’ll set up accounts for each of them (including invoicing), making it quick and easy for you.

What: Our video conferencing tool
We built Builder Meet to address the security concerns posed by using third-party tools available in the market – and to help us enhance our conversational AI models (see Natasha) at the backend. It offers seamless video communication within the Builder.ai ecosystem.

Our team can visually interact with you, or the experts working on your app, from their browser without any installation hassles. Builder Meet offers file sharing, real-time chat and screen share, all at a reduced cost.

What: Early warning system for project delays.
Health Check uses information from past projects within the Builder Knowledge Graph, to work out the probability of whether a current project will be delayed.
It also shows the most likely causes for the delay (for example, if a customer isn’t responsive and they’re taking longer to give feedback, or there are more bugs being raised by developers, or more complicated feature to feature interactions). We use these insights to take action ahead of probable issues and reallocate resources to prevent delays.
Tech used: Bayesian network inference – to provide the transparent, causal impacts of different factors on project delays.