> ## Documentation Index
> Fetch the complete documentation index at: https://braintrust.dev/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Generating beautiful HTML components
[Contributed](https://github.com/braintrustdata/braintrust-cookbook/blob/main/examples/HTMLGenerator/HTMLGenerator.ipynb) by [Ankur Goyal](https://twitter.com/ankrgyl) on 2024-01-29
In this example, we'll build an app that automatically generates HTML components, evaluates them, and captures user feedback. We'll use the feedback and evaluations to build up a dataset
that we'll use as a basis for further improvements.
## The generator
We'll start by using a very simple prompt to generate HTML components using `gpt-3.5-turbo`.
First, we'll initialize an openai client and wrap it with Braintrust's helper. This is a no-op until we start using
the client within code that is instrumented by Braintrust.
```typescript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import { OpenAI } from "openai";
import { wrapOpenAI } from "braintrust";
const openai = wrapOpenAI(
new OpenAI({
apiKey: process.env.OPENAI_API_KEY || "Your OPENAI_API_KEY",
})
);
```
This code generates a basic prompt:
```typescript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import { ChatCompletionMessageParam } from "openai/resources";
function generateMessages(input: string): ChatCompletionMessageParam[] {
return [
{
role: "system",
content: `You are a skilled design engineer
who can convert ambiguously worded ideas into beautiful, crisp HTML and CSS.
Your designs value simplicity, conciseness, clarity, and functionality over
complexity.
You generate pure HTML with inline CSS, so that your designs can be rendered
directly as plain HTML. Only generate components, not full HTML pages. Do not
create background colors.
Users will send you a description of a design, and you must reply with HTML,
and nothing else. Your reply will be directly copied and rendered into a browser,
so do not include any text. If you would like to explain your reasoning, feel free
to do so in HTML comments.`,
},
{
role: "user",
content: input,
},
];
}
JSON.stringify(
generateMessages("A login form for a B2B SaaS product."),
null,
2
);
```
```
[
{
"role": "system",
"content": "You are a skilled design engineer\nwho can convert ambiguously worded ideas into beautiful, crisp HTML and CSS.\nYour designs value simplicity, conciseness, clarity, and functionality over\ncomplexity.\n\nYou generate pure HTML with inline CSS, so that your designs can be rendered\ndirectly as plain HTML. Only generate components, not full HTML pages. Do not\ncreate background colors.\n\nUsers will send you a description of a design, and you must reply with HTML,\nand nothing else. Your reply will be directly copied and rendered into a browser,\nso do not include any text. If you would like to explain your reasoning, feel free\nto do so in HTML comments."
},
{
"role": "user",
"content": "A login form for a B2B SaaS product."
}
]
```
Now, let's run this using `gpt-3.5-turbo`. We'll also do a few things that help us log & evaluate this function later:
* Wrap the execution in a `traced` call, which will enable Braintrust to log the inputs and outputs of the function when we run it in production or in evals
* Make its signature accept a single `input` value, which Braintrust's `Eval` function expects
* Use a `seed` so that this test is reproduceable
```typescript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import { traced } from "braintrust";
async function generateComponent(input: string) {
return traced(
async (span) => {
const response = await openai.chat.completions.create({
model: "gpt-3.5-turbo",
messages: generateMessages(input),
seed: 101,
});
const output = response.choices[0].message.content;
span.log({ input, output });
return output;
},
{
name: "generateComponent",
}
);
}
```
### Examples
Let's look at a few examples!
```typescript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
await generateComponent("Do a reset password form inside a card.");
```
```
```
To make this easier to validate, we'll use [puppeteer](https://pptr.dev/) to render the HTML as a screenshot.
```typescript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import puppeteer from "puppeteer";
import * as tslab from "tslab";
async function takeFullPageScreenshotAsUInt8Array(htmlContent) {
const browser = await puppeteer.launch({ headless: "new" });
const page = await browser.newPage();
await page.setContent(htmlContent);
const screenshotBuffer = await page.screenshot();
const uint8Array = new Uint8Array(screenshotBuffer);
await browser.close();
return uint8Array;
}
async function displayComponent(input: string) {
const html = await generateComponent(input);
const img = await takeFullPageScreenshotAsUInt8Array(html);
tslab.display.png(img);
console.log(html);
}
await displayComponent("Do a reset password form inside a card.");
```
```
```
```typescript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
await displayComponent("Create a profile page for a social network.");
```
```
John Doe
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nulla ut turpis
hendrerit, ullamcorper velit in, iaculis arcu.
```
```typescript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
await displayComponent(
"Logs viewer for a cloud infrastructure management tool. Heavy use of dark mode."
);
```
```
12:30 PM
Info: Cloud instance created successfully
12:45 PM
Warning: High CPU utilization on instance #123
01:00 PM
Error: Connection lost to the database server
```
## Scoring the results
It looks like in a few of these examples, the model is generating a full HTML page, instead of a component as we requested. This is something we can evaluate, to ensure that it does not happen!
```typescript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
const containsHTML = (s) => /<(html|body)>/i.test(s);
containsHTML(
await generateComponent(
"Logs viewer for a cloud infrastructure management tool. Heavy use of dark mode."
)
);
```
```
true
```
Now, let's update our function to compute this score. Let's also keep track of requests and their ids, so that we can provide user feedback. Normally you would store these in a database, but for demo purposes, a global dictionary should suffice.
```typescript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
// Normally you would store these in a database, but for this demo we'll just use a global variable.
let requests = {};
async function generateComponent(input: string) {
return traced(
async (span) => {
const response = await openai.chat.completions.create({
model: "gpt-3.5-turbo",
messages: generateMessages(input),
seed: 101,
});
const output = response.choices[0].message.content;
requests[input] = span.id;
span.log({
input,
output,
scores: { isComponent: containsHTML(output) ? 0 : 1 },
});
return output;
},
{
name: "generateComponent",
}
);
}
```
## Logging results
To enable logging to Braintrust, we just need to initialize a logger. By default, a logger is automatically marked as the current, global logger, and once initialized will be picked up by `traced`.
```typescript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import { initLogger } from "braintrust";
const logger = initLogger({
projectName: "Component generator",
apiKey: process.env.BRAINTRUST_API_KEY || "Your BRAINTRUST_API_KEY",
});
```
Now, we'll run the `generateComponent` function on a few examples, and see what the results look like in Braintrust.
```typescript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
const inputs = [
"A login form for a B2B SaaS product.",
"Create a profile page for a social network.",
"Logs viewer for a cloud infrastructure management tool. Heavy use of dark mode.",
];
for (const input of inputs) {
await generateComponent(input);
}
console.log(`Logged ${inputs.length} requests to Braintrust.`);
```
```
Logged 3 requests to Braintrust.
```
### Viewing the logs in Braintrust
Once this runs, you should be able to see the raw inputs and outputs, along with their scores in the project.
### Capturing user feedback
Let's also track user ratings for these components. Separate from whether or not they're formatted as HTML, it'll be useful to track whether users like the design.
To do this, [configure a new score in the project](/annotate/human-review#configure-review-scores). Let's call it "User preference" and make it a 👍/👎.
Once you create a human review score, you can evaluate results directly in the Braintrust UI, or capture end-user feedback. Here, we'll pretend to capture end-user feedback. Personally, I liked the login form and logs viewer, but not the profile page. Let's record feedback accordingly.
```typescript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
// Along with scores, you can optionally log user feedback as comments, for additional color.
logger.logFeedback({
id: requests["A login form for a B2B SaaS product."],
scores: { "User preference": 1 },
comment: "Clean, simple",
});
logger.logFeedback({
id: requests["Create a profile page for a social network."],
scores: { "User preference": 0 },
});
logger.logFeedback({
id: requests[
"Logs viewer for a cloud infrastructure management tool. Heavy use of dark mode."
],
scores: { "User preference": 1 },
comment:
"No frills! Would have been nice to have borders around the entries.",
});
```
As users provide feedback, you'll see the updates they make in each log entry.
## Creating a dataset
Now that we've collected some interesting examples from users, let's collect them into a dataset, and see if we can improve the `isComponent` score.
In the Braintrust UI, select the examples, and add them to a new dataset called "Interesting cases".
Once you create the dataset, it should look something like this:
## Evaluating
Now that we have a dataset, let's evaluate the `isComponent` function on it. We'll use the `Eval` function, which takes a dataset and a function, and evaluates the function on each example in the dataset.
```typescript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import { Eval, initDataset } from "braintrust";
await Eval("Component generator", {
data: async () => {
const dataset = initDataset("Component generator", {
dataset: "Interesting cases",
});
const records = [];
for await (const { input } of dataset.fetch()) {
records.push({ input });
}
return records;
},
task: generateComponent,
// We do not need to add any additional scores, because our
// generateComponent() function already computes `isComponent`
scores: [],
});
```
Once the eval runs, you'll see a summary which includes a link to the experiment. As expected, only one of the three outputs contains HTML, so the score is 33.3%. Let's also label user preference for this experiment, so we can track aesthetic taste manually. For simplicity's sake, we'll use the same labeling as before.
### Improving the prompt
Next, let's try to tweak the prompt to stop rendering full HTML pages.
```typescript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
function generateMessages(input: string): ChatCompletionMessageParam[] {
return [
{
role: "system",
content: `You are a skilled design engineer
who can convert ambiguously worded ideas into beautiful, crisp HTML and CSS.
Your designs value simplicity, conciseness, clarity, and functionality over
complexity.
You generate pure HTML with inline CSS, so that your designs can be rendered
directly as plain HTML. Only generate components, not full HTML pages. If you
need to add CSS, you can use the "style" property of an HTML tag. You cannot use
global CSS in a