AI – IMG.LY Blog

Evaluating CoDesign MCP performance in resizing

Dustin — Fri, 21 Aug 2026 09:00:00 GMT

You designed a banner. The same day, you need it in five sizes, and none of them can be wrong.

We handed that job to six AI models through CoDesign MCP: take this approved design and rebuild it for five ad formats. All six came back with ads you could ship. Gemini 3.7 Flash did it for thirty cents and passed every check we wrote.

That is what CoDesign MCP is for. You point an AI assistant such as Claude at your approved design, name the formats you need, and it opens the file, rebuilds the layout for each one and exports production files. What comes back is an editable design file, not a picture of one.

Speed and price are the easy things to measure. We wanted to measure quality too, so we built an eval that runs the job automatically, opens every result and reports what passed. This is the first scenario out of that suite.

Three banners, one brief

Three different models were handed the same 320×50 slot, cut from the same master creative against the same brief, and all three came back with an ad you could traffic tomorrow. Every one of them keeps the wordmark and the button, nothing is clipped, and no type is set below 11px.

The 300×250 they came from carried five things: the wordmark, the headline, a retention chart, the offer line and the button. At 320×50 the wordmark and the button survive and the chart cannot, which leaves room for exactly one line of copy, so each model had to choose between the headline and the offer.

One dropped the headline and kept the offer, “14-day free trial”.
One kept the headline but cut “your users” down to “users” to buy itself the line.
One kept the headline whole across two lines and dropped the offer.

We have opinions about which of those reads best, the way any design team would. What we do not have is a way to prove one of them correct, because the choice depends on whether this campaign is selling the product or the trial, and that lives in the media brief rather than in the design. Our checks can confirm that all three are valid and that none of them are broken; they cannot tell you which one the campaign actually needed. Grading the half that has a right answer and leaving the other half to the people who own the brief is the distinction our evals are built around.

What is gradeable and what is not

The obvious thing to evaluate is the thing that demos well, prompt in and poster out, but we were more interested in the work that actually fills a designer’s week, which is mostly derivative. The design decisions were made once, at the master, and everything after that is a matter of carrying them faithfully into thirty other shapes. That is more technical work than creative work, which is exactly why it is worth handing to an agent.

Ask which of the finished designs is better and you have a question with no ground truth to appeal to, since human raters disagree with each other and LLM judges reliably prefer their own output. You could ask a lot of humans and rank the results, but that is not something you can run on every build. Wrong designs, though, are wrong in ways you can query: text sitting behind another block, content off the page, type under the legibility floor. Part of design quality is judgement and part of it is fact, and what we have built grades the second part, which means everything below is an automated result rather than a design review. A model can pass every check we wrote and still make an ugly ad.

What we are doing right now

When a model finishes a scenario, programmatic checks open the result and ask whether it is plausibly good. That starts with whether the run produced every design artifact and .imgly file at the right sizes, and the rest depends on the scenario: in a localization run we also check that the words were actually replaced.

This is a technical floor rather than a grade. LLM judges are still unreliable enough that we do not use one on output quality, because if it is hard for us as designers to say which design is more correct, it is harder for a model, and the result would be noise.

In the future we plan to experiment more with such methodologies but we also do not want to rely on the false security of an extremely noisy grading process.

The comparison view for the display-banner matrix: cost and time sitting directly above what each model actually produced. The 1/1 counts runs, one repeat per cell, not check scores.

One master, five ad sizes

The scenario is a display campaign, because that is the version of this problem customers actually have: the same size matrix, every campaign. The master is a 300×250 MPU for ATLAS, an imaginary product-analytics SaaS running a free-trial campaign, carrying a wordmark, the headline “See what your users actually do”, a retention bar chart labelled “43% retained”, the line “14-day free trial. No card.”, and a “Start free trial” button.

The 300×250 master at the top, and the four other sizes on the media plan cut from it. ATLAS is our own fixture, not a real customer.

The brief asks for the full matrix: 300×250, 728×90, 160×600, 300×600 and 320×50. Those sizes span a 16:1 swing in aspect ratio, so the right answer at each one is a genuinely different composition, and at some of them it means taking content out rather than shrinking it.

Ten checks grade the result and two of them do the real work. 728×90: the chart was dropped, not shrunk kills the model that squashes a bar chart into a 90-pixel strip, which satisfies every geometric constraint in the brief while being completely wrong. the five sizes are different layouts kills the model that scales one composition five ways and calls it a matrix.

The master first, then the six 728×90 leaderboards. Every model that delivered deleted the retention chart.

Every model that delivered arrived at the same composition, wordmark left, headline across the middle, offer underneath it, button on the right and the retention chart deleted, which is six models from five different vendors reaching the same answer independently. We did not expect that, and it is still the result we find hardest to stop thinking about.

The same six at 160×600, master first. With vertical room they all kept the chart. The flat format is where the decision to drop it had to be made.

Which model is best for using with CoDesign MCP?

#	model	checks	model spend	time	tool calls
1	gemini-3.7-flash	10 / 10	$0.30	6.7 min	37
2	glm-5.2	10 / 10	$0.40	8.2 min	25
3	gpt-5.6-sol	10 / 10	$0.52	4.0 min	35
4	grok-4.6	10 / 10	$1.20	20.9 min	39
5	claude-fable-5	10 / 10	$3.95	10.3 min	26
6	claude-opus-5	10 / 10	$3.97	18.8 min	40

Running this scenario illustrates general numbers across our eval suite. The 6 models above are consistently able to follow the prompts to produce different output designs as requested in the scenario. These numbers also highlight quite the spread in model costs and time spend. What you do not see in these numbers is that glm 5.2, gpt-5.6-sol and grok 4.6 are producing suboptimal designs in our opinion. They are valid, but subjectively fable, opus and gemini 3.7 consistently across runs and across scenarios perform simply better.

The biggest surprise for us is Gemini 3.7 Flash, which produced the whole five-size matrix for $0.30 in model spend, in under seven minutes. At thirty cents a designer opens six composed variants instead of a blank artboard and takes the best one further, which changes what the tool is for.

Meanwhile there were no surprises regarding Opus 5.0 and Fable 5.0, both models perform really well - but are slow and expensive.

What does it mean for CoDesign MCP

You can now resize a master design into 5 different sizes reliably for $0.30 using Gemini 3.7 Flash and the CoDesign MCP.

If you are on a Claude Subscription plan, you might just want to continue using your Claude Code with the CoDesign MCP since it offers the best overall design quality - even if it’s slow and somewhat expensive.

We will continue to work on upgrading and improving measurements of how well CoDesign MCP works with different models, looking into other scenarios like localization (translating and changing a design for a different locality and language) as well as rebranding existing design assets.

You can install CoDesign MCP at img.ly/codesign.

AI Design Agents in 2026: Which One Fits Your Work

Jan — Mon, 17 Aug 2026 09:00:00 GMT

Most roundups of AI design agents rank the tools one to six. That only works if every reader wants the same thing. A developer wiring up app screens and a brand manager producing a campaign are not shopping in the same market, even though the same six products keep showing up in both their search results.

This survey sorts by the work instead. Two questions do most of the sorting: what you are making, and what you are left holding when the agent stops. If you want the wider tool category rather than agents specifically, we have compared the AI design tools built around prompt-first workflows.

We ran five of the six ourselves, on the same two prompts, and the findings below are what we saw rather than what the marketing pages say.

Disclosure up front. IMG.LY publishes this survey, and two of the six tools here have a commercial relationship with us: CoDesign is our own product, and Manus is an IMG.LY customer. Both entries say so where they appear, we assess both against the same criteria as everything else, and anything we did not test ourselves rests on public sources.

What counts as a design agent

Three tests separate agents from the wider AI design tool space. The tool must plan multi-step work from a single goal. It must execute design operations rather than only advising. Its output must be usable design work: files, screens, or code.

These tests exclude two familiar groups. Image generators such as Midjourney and Ideogram produce strong single images without planning or executing a task around them. Chat assistants configured with design prompts, such as Taskade’s design agent templates, return recommendations in text. Both are useful, but neither does design work on your behalf.

Six products meet that definition in August 2026. We work through the definition in more detail in What Is an AI Design Agent?

How we tested

One prompt per category, given identically to every tool in that category.

For the interface tools: design a three-screen onboarding flow for a habit-tracking app, the screens in order being a welcome screen, a goal-setting screen, and a first habit formation screen, styled clean, high contrast, large type, one accent color.

For the graphic design tools: a launch campaign for a cold brew called Northwind, in three formats, an Instagram post, a story, and a DIN A5 flyer, warm and minimal, cream background, one product photo, headline supplied.

Two things decide it, and neither is how good the first draft looks.

Is the agent context aware? Can you feed it data, make it familiar with your brand, and does it hold that across every variant? We do not much care how fancy the design is. The table stakes have to be right before anything else counts.

And does it fit the habits you carried over from the before-AI age? Making a small revision by going back and prompting an AI again is the wrong shape. You want to be the human in the loop who makes that edit manually, because by the time you are prompting for the third time your frustration is already high enough that you have stopped wanting to.

One caveat on the Figma run, since it cuts against us: it executed the prompt inside IMG.LY’s own design system, so its icons, fonts, and patterns inherit work our designers had already done. That makes the head-to-head somewhat unfair in Figma’s favor.

Manus is the exception. We have not run it hands-on, and its entry below rests on public sources.

What you hold afterward

Generation quality converges fast. Every tool here produces a credible first draft, and the differences narrow every quarter. If you want to see where the underlying models differ, we benchmark 15 of them against the same design prompts.

What you keep does not converge. Some tools hand you a file inside an editor where you can select any element and change it. Others give you an export to finish in a different program. The rest keep the work inside their own platform, editable there and nowhere else.

AI design agents for app and web interfaces

Screen design has the most mature agent tooling, partly because UI has structure a model can reason about and partly because the output can be code.

Google Stitch

Stitch is a free experimental tool from Google Labs that generates multi-screen mobile and web interfaces, plus frontend code, from text or image prompts. It launched at I/O 2025 and gained Gemini 3 and interactive prototype flows in December 2025.

You describe an app and Stitch plans the screens, which you refine through further prompts. Screens render in the browser, with code export for the web stack and a paste-to-Figma path that preserves layers. Stitch has no source format of its own, so you can only keep editing in the exported code or the Figma file it hands off to. Reviewers recommend exploring in Stitch and refining elsewhere.

On our onboarding prompt it did not adhere to the brief. Habit formation came before goal setting, so the screen order was wrong. Button styles were somewhat inconsistent, the look and feel differed between screens, visible in the progress indicator on the welcome screen against the one on goal setting, and the aspect ratios were weird.

Editing is where it separates from Figma. Double-clicking does let you change the text, but you are apparently supposed to edit with AI, which is a nuisance when you only want to make a few design fixes. The brand kit is the part we liked: colors, accent colors, fonts, and spacing can all be changed globally. Though it looked like Stitch did not adhere to its own system that well.

It is free while in Google Labs. Google does not publish the generation caps, and third-party sources describe the limits inconsistently. Best fit is getting a first version on screen fast, especially for developers who want screens and starter code in one pass. Its Labs status makes it a tool to explore with rather than build a production process on.

Figma Agent

Figma’s agent works directly on the design canvas. It entered beta in May 2026 and gained custom skills, web search, and external MCP connections at Config in June 2026.

You prompt from any layer. The agent executes multi-step tasks: bulk edits across screens, dark-mode conversion, populating designs with real content, turning feedback threads into revisions. Custom skills written as markdown steer how it works.

Output is native Figma layers with components, variables, and design tokens preserved. This is the strongest design-system integration of the six, because the agent works inside the system your team already maintains rather than approximating it. Code generation routes through Figma Make, a separate product.

On our onboarding prompt it executed almost perfectly. The design was very clean, it adhered to the standards of modern apps, and it almost looked like something you would find browsing the App Store. Even the copy was decent, and it did not read like AI slop. The screens looked thought through: asked for a primary goal, it offered routines, breaking a bad habit, and staying consistent, which are the broad categories without too many assumptions on top. It stuck to the screen order, it baked in some interactivity, and it caught small details like “remind me every day at 8 a.m.” on the habit formation screen. As a novice you could take this and start prototyping.

Remember the caveat above, though: this run inherited IMG.LY’s design system, so the icons, fonts, and patterns were already good before the agent started.

The agent is free during beta, and Figma has said AI credits will apply at general availability without publishing amounts. It requires a full seat on Professional, Organization, or Enterprise; full seats start at $16 per person per month on the annual Professional plan. Early reviews report rough edges, including responsive layouts that broke on mobile.

In the head-to-head, Figma beat Stitch by a wide margin, which was to be expected.

Best fit is product and UI teams already working in Figma with an established design system.

AI design agents for marketing and brand assets

In campaign work you are producing dozens of assets at once, they all have to stay on brand, and some of them have to survive contact with a printer.

Lovart

Lovart is a dedicated design agent that turns a brief into a coordinated set of deliverables: logos, posters, packaging, social assets. It launched publicly in July 2025 after a closed beta that drew several hundred thousand users.

Give it a campaign-level brief and it analyzes intent, researches references, then generates dozens of assets sharing one visual system. You and the agent iterate on ChatCanvas, an infinite canvas where you both edit the same design through conversation. Results land as layers with typography kept separately, and Lovart offers targeted element and text edits in place.

On the Northwind campaign it was fairly quick, and the result was very aesthetically pleasing. We had very little to quibble with. The design subtleties were there, the product placement, the coffee beans, some cloth, and it stuck very well to the specification. Exactly what we had in mind.

The one quibble is brand drift. Look at the logo, look at the bottle, and there is significant drift in brand identity across the three formats. That is fixable in a real scenario, because you can link a brand, so you would have the product image and the logo ready and we would not expect much trouble then, whether you use Lovart to create variations, produce marketing material to A/B test, or adjust for different formats.

Editing is the problem. Ask to change the headline text and you cannot tell whether you are even looking at the same font. Quick edit means writing another prompt. There are no layers, and there is nothing you can do directly. So while the output looks nice, for revision work it is effectively useless.

Reviewers report that precise pixel-level work tends to move into Photoshop or Figma anyway, which matches what we found.

Pricing runs on credit tiers from Starter to Ultimate, 2,000 to 27,000 credits per month. Lovart renders prices dynamically and third-party reports of the dollar amounts vary between roughly $15 and $90 monthly, so check the pricing page directly. Credit consumption is hard to predict on agent-driven tasks.

Best fit is marketing teams whose unit of work is the campaign rather than the asset, and who accept a finishing pass elsewhere.

Canva AI Assistant

Canva’s assistant plans and executes design tasks by calling Canva’s own tools. It launched in April 2025 and received a tool-calling rebuild, announced as a research preview, in April 2026.

It runs multi-step jobs such as producing a multi-channel campaign, and pulls context from connected sources including Slack, Gmail, Google Drive, and Notion. Scheduled tasks run in the background, with finished work arriving as drafts for review.

Output is layered, fully editable Canva designs across presentations, social posts, documents, and spreadsheets. You can change any element without regenerating. The constraint is Canva itself. The work stays editable inside Canva, and getting it out means exporting.

On the same Northwind prompt it was fairly quick, and it appears to match the brief against the vast template library Canva already has. From the get-go, though, it did not really stick to the specification. We asked for a cream background and one product photo. There is no cream background on the first asset, and we do not know why the product would be displayed on a laptop, which is plain weird. Only the last of the three is anywhere near acceptable.

We also had no indication of whether it generated variations or ignored that part of the brief. Opening one of the designs gives you more variations, including the one you did not select. What you do not get is a multi-page layout you can change, or any way to specify changes precisely. At that point you are simply inside Canva, editing.

As a starting point we would have been just as well off picking one of the Canva templates, uploading the product image, and adjusting the text. Compared with Lovart on the identical brief, Canva really does fall short.

The assistant is available on the free tier with rate limits and monthly credits. Pro is $18 monthly. Business is $25 per user monthly with larger allowances. Reviewers warn that letting the assistant run a job end to end uses premium credits faster than editing by hand.

Best fit is teams already on Canva who want campaign production handled conversationally with brand kits applied automatically.

IMG.LY CoDesign

Disclosure: CoDesign is made by IMG.LY, which publishes this survey.

CoDesign is a free local MCP server that gives an agent you already use a full design engine. It entered technical preview in June 2026.

Instead of going to CoDesign, you install it into your own agent with one command, and the agent then performs design operations in conversation with you. IMG.LY documents the install for Claude Code and Codex, and any client that can run an MCP server locally works the same way. CoDesign can generate, rebrand, resize, edit, localize, import, and judge, the last checking a design against your brand rules. A representative chain: import a PSD, rebrand it, resize it to a dozen formats, check it against the rules, export.

Output is a structured scene rather than a flat image. It opens in an editor where you select any element and change it without regenerating anything. Underneath is CE.SDK, IMG.LY’s production design SDK. It imports PSD, IDML, PDF, and PPTX, and exports PDF with CMYK, bleed, and ICC profiles when a printer needs them. The agent, your code, and a human editor all work on the same file.

We ran the Northwind prompt through Claude Code. Before touching the canvas it asked where the product photo should come from, what accent color to use, what kind of cream we meant, and it drafted the copy for approval. In normal use you would run this inside your marketing documentation and brand assets, so your coding agent already carries a pile of context and can answer most of that itself. We gave it no brand context at all, to keep the comparison fair with the others.

It left us waiting a bit longer, though that is not quite fair to say, since it works in tandem with the coding agent. It was thorough and diligent. It ran a self-check against a set of axes before handing the design back, and because we supplied no brand context those came back blank. It passed, and returned one editable master file.

When it needed the product image it opened the IMG.LY dashboard, where the AI Gateway picked the model for it and credited the generation, which was frictionless.

The design that came back is fairly conservative and does not assume too much: some copy, “now pouring, limited first batch”, a CTA. Fairly bare bones, but it works. What matters is that it stuck perfectly to the specification. The square asset, the story asset, and a PDF carrying its color space and a resolution we could hand to a printer. Edit one element, tell it to regenerate, and it propagates: same colors, same logo, same product image across every variant.

All three formats came out of a single master file, generated in one pass from the same brief:

The local server is free to install and run with no account needed to start. An optional free account adds AI image generation, which routes out through IMG.LY’s AI Gateway. Paid licensing applies when you host the server for other people or embed it in a product.

Best fit is anyone running an agent-first local workflow who needs production-grade editing, print-ready files, and brand consistency across every variant. As a technical preview it is early, and behavior can change between releases.

Delegating design inside a larger task

Manus

Disclosure: Manus is an IMG.LY customer. This is also the one entry we did not run ourselves, so unlike the five above it rests on public sources rather than hands-on testing.

Manus is a general-purpose autonomous agent with a design workspace inside it. Design is one capability among research, app building, and document production.

Manus decomposes a goal into subtasks, browses for context, runs for long stretches without supervision, and continues while you are offline. For design work it researches before designing, then produces assets you refine through object-level edits: click an element to change colors, swap a background, or reshape it, without regenerating the whole asset.

Output covers images, video, 3D assets, and presentations. Its public documentation does not describe a layered source file of the kind a dedicated design tool exposes, so you edit objects rather than a design file. General agents are closing the gap on design output faster than anything else in this survey.

The free tier includes 300 daily refresh credits. Paid plans run $20, $40, and $200 monthly for 4,000, 8,000, and 40,000 credits. Reviews describe unpredictable credit burn on complex tasks.

Best fit is founders and operators who want one agent handling research, documents, and adequate design output, and who value autonomy over design-tool depth.

Which tools leave you a real editor

Three tools in this survey give you a real editor after generation. Figma Agent puts native layers on the Figma canvas. Canva’s assistant produces fully editable Canva designs. CoDesign returns a structured scene that opens in a full editor. In all three you select an element and change it, instead of rewriting the prompt and hoping the next version keeps the parts you liked.

Lovart sits between. It has a canvas and in-place editing, and precision work still tends to finish in Photoshop. Stitch hands off to Figma or to code. Manus edits at the object level, which suits an agent whose remit is much wider than design.

Revision takes longer than the first draft, and tools that keep you in control of the file absorb those cycles.

Destination app or capability in your stack

Five of the six tools here are destinations. You open Figma, Canva, Lovart, Stitch, or Manus, do the work there, and take the result away. That model is familiar and it works.

The sixth, CoDesign, is a design engine that installs into the agent you already use, so the design step happens inside your existing workflow.

That split is narrower than it first appears. Figma opened an MCP server in March 2026, so external coding agents can drive its canvas. Canva shipped an MCP server in February 2026 that exposes design generation inside ChatGPT, Claude, and Microsoft Copilot, and says more than 12 million designs have been created that way. Being reachable from your agent is not unique to CoDesign.

The tools differ in what they hand back. Drive Figma’s MCP and you get a Figma file, which is useful if your team lives in Figma. Drive Canva’s and you get a Canva design. Drive CoDesign and you get a portable scene file, PDF with CMYK and bleed, PSD and IDML import, and no platform you have to keep an account with. For teams whose output ends at a printer or inside their own product, that portability decides it. It matters much less if your work already lives in Figma or Canva.

How to choose

Start with what you are making. UI and app screens point to Stitch for speed or the Figma Agent for depth. Campaign and brand assets point to Lovart for volume from one brief, Canva for teams already there, or CoDesign for output that has to stay editable and reach print. If design is a side task inside something larger, Manus covers it.

Then check the exit path before you commit, because it is the part you cannot change later. Ask where the file lives, whether you can edit it without regenerating, and what happens when it needs to leave the tool.

Frequently asked questions

What is an AI design agent?

An AI design agent plans and executes multi-step design work from a goal, producing usable design output such as files, screens, or code. What Is an AI Design Agent? goes into where the line falls.

How is an AI design agent different from an image generator?

An image generator returns a picture for a prompt. An agent plans a sequence of steps, performs design operations, and produces work you can carry forward.

Are AI design agents free?

Three of the six have a free way in: Stitch while it is in Labs, Manus on its free tier, and CoDesign’s local server. Figma’s agent is free during beta but needs a paid seat. Lovart and full Canva use are subscriptions.

Do AI design agents replace designers?

No. Every tool in this survey keeps a person reviewing the output, and all six advertise editing after generation.

Which AI design agents produce editable files?

Figma Agent, Canva’s assistant, and CoDesign keep element-level editability in a design file. Lovart offers in-canvas editing, though external finishing is common. Manus edits objects in place. Stitch relies on its Figma and code exports.

Can I use a design agent inside Claude or ChatGPT?

Yes, through MCP. Canva exposes design generation in ChatGPT, Claude, and Copilot. Figma opened its MCP server to external coding agents. CoDesign runs as a local MCP server in coding agents such as Claude Code and Codex.

Introducing IMG.LY AI Benchmarks

Jan — Mon, 03 Aug 2026 00:00:00 GMT

IMG.LY GenAI Benchmarks is live. 15 text-to-image models, 37 prompts taken from real design jobs, three fixed seeds each, 1,662 generated images, roughly 7,400 scores. Every prompt, every score, every generated image and the full methodology are public.

Why we ran it

Our customers embed image generation in products that ship physical, commercial output. Web-to-print storefronts, merch platforms, campaign tooling, template systems. When they asked us which model to use, the honest answer was a shrug and a link to a leaderboard that measures something else.

The existing arenas rank aesthetic preference by crowd vote. That’s a signal, sure, but not the one most relevant to these use cases. They want to know whether the image has a real alpha channel, or whether users got the hex they asked for and whether a word is spelled right. A 10,000-piece personalization run needs to hold the character across every variant. Those are the failures that cost you a reprint or a support ticket, and none of them show up in a preference ranking.

So we built the one that answers our customers’ question. Which model should you build on for production design work.

Why some of it is judged, not measured

Plenty of what matters here is measurable, and we measure it. Native output size, wall-clock latency and price per image come straight off the run. Transparency is a file-level check for a real alpha channel rather than a painted-on white rectangle. Color accuracy is the CIEDE2000 distance between the hex the prompt required and the color that came back.

The rest needs a judgment call. Whether an image satisfies “three blue cubes stacked on the left and one red sphere on the right” is a checklist, and something has to look at the picture and tick it off. Whether a banner leaves a clean area for a headline has a right answer, but not a computable one.

For those criteria a vision model does the judging, against a checklist written per prompt before the run. We added two rules to make the VLM’s judgment more transparent and control for arbitrariness. Every judgment records a written rationale beside the score, so you can read why an image lost a point and tell us we’re wrong. The criterion that really does need a person, design-readiness, the “would a professional ship this with under five minutes of cleanup” question, is surfaced to a human in the loop (that would be yours truly, the author).

The rest of the protocol is the boring part that makes the numbers worth anything. Identical prompts for every model, default parameters, fixed seeds, no per-model prompt tuning ever, a versioned suite, and published scores that are never quietly restated. Partner models get no special treatment, and the Gateway links stay well away from the rankings.

Why we split it by use case

Average our eight criteria with equal weight and Nano Banana 2 Lite leads at 3.87 out of 5, while last place, at 2.55, goes to Nano Banana Pro, the most expensive flagship in the set. That’s not a data error. Nano Banana Pro has the best brand-color fidelity in the field and renders every required string in the suite exactly. It also costs 28 times more per image than the cheapest model and takes 23 seconds. An unweighted average treats all of that as equally important, so it buries the model with the best output.

This is the chart everyone asks for, and its vertical axis is the number you should not rank on. Explore the live, interactive version, where every point links to that model’s scores.

Which is what leaderboards do. So we publish the same measurements per use case as well, with a weight matrix per job and the reasoning behind each matrix written out. Print files weight resolution and color heavily, because a 1024px generation is a 3.4-inch print and an out-of-gamut red is a reprint. Personalization at scale weights cost and adherence, because unit economics decide the run. Merch weights the alpha channel above everything else, because a sticker without a clean cutout isn’t a sticker.

Same numbers, different question, different answer. That’s why the section opens by asking what you’re building instead of handing you a ranked list.

The result that surprised us most

Reweighting moves models the length of the table.

GPT Image 1.5 sits twelfth of fifteen on the unweighted blend, dragged down by cost and latency. Reweight for merch and stickers, where transparency carries 35 percent of the matrix, and it wins outright at 3.72, nearly a full point clear of second place. It’s the only model in the run that reliably returns a genuine alpha channel. Thirteen of the fifteen return none at all, and a few paint a fake checkerboard into the pixels, which looks like transparency right up until it reaches a printer.

On a general leaderboard, the best model for one of our customers’ most common jobs reads as a mid-table also-ran. That’s the argument for the whole project.

The typography column surprised us a second time. Ideogram has the strongest public reputation for rendering text, and in our run it ranks twelfth of fifteen on measured text accuracy, behind several general-purpose flagships that now render every required string in the suite exactly. Route typographic work to a specialist on reputation alone and you can end up with worse type than the model you already call by default.

Two more capability gaps got written up as standalone findings. Brand color, where the best model manages 3.67 out of 5 and most of the field sits below 3. And consistency, where no model holds a described character across three scenes and the ceiling is 3.89.

What this means for an editor

We build editors, so our conclusion isn’t neutral. The data still points where it points.

Read the four findings together and they describe the same problem four times. The cut-out is missing, the color is close but not the color in the brand book, the headline is right until it isn’t, and the character drifts between scenes. None of these are bugs waiting on the next model release. They’re what generation is, a probabilistic first draft. Getting from that draft to something a customer can order takes a handful of deterministic corrections, and somebody has to be able to make them.

That’s an editor’s job. Background removal and edge cleanup close the transparency gap. Brand kits and exact color values close the color gap. Editable text layers turn a wrong character into a two-second fix instead of a regeneration. Reusable assets and templates carry identity that regeneration won’t. Our AI Editor gives your users those controls over whatever model produced the image, and the AI Gateway makes routing per job, which the data says you should be doing, a config change instead of another integration.

We’d rather make that case with numbers anyone can check, including the ones that make models we partner with look bad.

Caveats, and what’s next

This is a pilot dataset. Quality criteria are judged by a vision model, not yet by the blind expert panel, and design-readiness isn’t scored at all. The weight matrices are provisional. When the frozen suite lands, scores restate once and the suite version changes; after that, nothing gets restated without a clearly labeled new methodology version.

The roster will grow. Image editing and video are the obvious next modalities, and we plan to re-run the suite whenever a notable model ships.

Go pick a use case and see which model wins your job. If the numbers disagree with your own experience, the prompts and the raw images are all sitting there, and we’d like to hear about it.

The Best GenAI Models for Web-to-Print: A Buyer's Guide

Jan — Fri, 26 Jun 2026 00:00:00 GMT

No single generative model is “best” for web-to-print. The right pick depends on the specific job, and print has jobs that screen-first leaderboards don’t measure. A model that tops a general image-quality benchmark can still be useless for print if it can’t render legible text, can’t output vector, or shifts hard out of CMYK gamut.

This guide gives you the criteria that actually matter for print, a scoring rubric to apply them, and a model-by-model read on where each one wins. The goal is to route each task to the model that fits it, which is why a model-agnostic (“bring your own model”) integration matters more than any one provider’s marketing.

How to read this guide. The criteria and rubric are the durable part. The model roster moves monthly: treat the specific model notes as a snapshot, and re-run the rubric against current versions before you commit. The per-model benchmark scores come from IMG.LY’s GenAI benchmark suite, which is now live: a pilot dataset you can explore today, re-run whenever new models drop. The gated PDF report that packages the weighted per-use-case totals and the full test set is the piece we’re opening to early-access subscribers first (sign-up at the end of this guide).

Why print needs its own criteria

A web-to-print product sells a physical object. That changes what “good” means:

The output is measured at 300 DPI on a substrate, not at 72 PPI on a retina screen.
Color is CMYK and spot, not sRGB.
Logos and type must scale and stay crisp: vector, not pixels.
The customer is a non-designer, so the model has to be controllable enough to stay inside a template.
And because it’s a commercial product sold to a customer, the licensing of the output carries real legal exposure.

General benchmarks (LMArena-style human preference, aesthetic scores) miss most of this. The eleven criteria below are built around it.

The eleven evaluation criteria

1. Output type and format: raster vs. vector

Why it matters for print: Logos, icons, line art, and type need to scale to any size and print crisply. Raster can’t; vector can. A model that outputs (or can be cleanly traced to) native SVG / vector is uniquely valuable for the print-specific elements that screen apps don’t care about.

What to test: Can it produce vector natively? If not, how cleanly does its output vectorize? Does the vector survive into PDF/X as real paths?

2. Resolution and maximum dimensions

Why it matters: A ~1024px generation is ~3.4” at 300 DPI. Large-format and even A4 need far more. Native max resolution, and how well the model holds detail when upscaled, determines whether you can print the output at size without a quality penalty.

What to test: Native max output dimensions; detail retention at 4×/8× upscale; effective printable size at 300 DPI.

3. Visual quality and fidelity

Why it matters: The baseline. Realism, coherence, absence of artifacts, anatomical and structural correctness. Crucially, how it holds up under print’s unforgiving close inspection: banding, mushy detail, and plasticky textures show more on paper than on screen.

What to test: Side-by-side preference on print-representative prompts; artifact rate; detail at print scale.

4. Text and typography rendering

Why it matters: Print is full of words, and most models garble text inside images. For anything where text-in-image is unavoidable (a generated poster, a label motif), legibility and kerning are make-or-break. Best practice is still to set real type as a separate layer, but some jobs need it baked in. Our text-reliability finding measures exactly this. The top of the field has largely solved Latin headlines: three models rendered every required string in our suite exactly. Small type and non-Latin scripts have not been solved, and the model most famous for typography ranks twelfth of fifteen. One wrong character still means regenerating the whole image, which is the real argument for keeping type on its own layer. Compare every model on the same wordmark prompt to see the spread.

What to test: Legibility of short and long strings; correct spelling; kerning; multi-line; non-Latin scripts.

5. Prompt adherence and controllability

Why it matters: Non-designers need predictable results, and templates need the output to land where it’s told. Controllability spans prompt adherence, image-to-image / inpainting / outpainting, style and reference-image conditioning, and seed stability for repeatable results.

What to test: Does it honor constraints? Quality of edits (generative fill, expand, object removal)? Reference-image fidelity? Same-seed reproducibility?

6. Color accuracy and CMYK-readiness

Why it matters: Vivid RGB outputs clip hard in CMYK and shift on press. Models that stay closer to printable gamut, and outputs that convert predictably, mean fewer surprises and less soft-proofing friction. Even before the CMYK conversion, hitting an exact brand hex in RGB is hard: our brand-color finding measures color accuracy with CIEDE2000 against the required hex, and no model reliably lands the color in your brand book.

What to test: Gamut coverage vs. CMYK; shift magnitude after ICC conversion; behavior on brand-critical colors and near-whites/blacks.

7. Transparency and background handling

Why it matters: Merch, stickers, die-cuts, and product compositing need clean subjects on transparent backgrounds and accurate edges (hair, glass, fine detail). Native transparent output and clean cutout edges save a manual masking step. In the models themselves this is close to unsolved: our transparency finding shows 13 of 15 tested models emit no real alpha channel at all, which is exactly why a background-removal step belongs in the pipeline rather than a prayer that the model returns a clean cutout.

What to test: Native alpha output; edge quality on hard cases; cutout-path readiness.

8. Responsiveness / latency

Why it matters: Two very different bars. In an interactive editor, anything over a few seconds breaks flow. In a batch/VDP run, throughput and concurrency matter more than per-image speed. The right model differs by context.

What to test: P50/P95 latency at your resolution; throughput under concurrency; cold-start behavior.

9. Cost per generation

Why it matters: Unit economics. A few cents per image is invisible for interactive one-offs and brutal across a 100k-recipient VDP run. Price has to be weighed against quality for the specific job. You don’t pay flagship prices for a draft thumbnail.

What to test: Cost per image at your resolution/step settings; cost of edit vs. generate; volume pricing.

10. Consistency and repeatability

Why it matters: Brand work needs the same input to yield the same (or controllably similar) output, and product likenesses must not drift. Seed control, character and style consistency, and low variance separate “brand-safe” models from “slot-machine” ones. Our consistency finding puts numbers on the drift: no model holds a described character across scenes, so identity has to come from templates and editing, not from a regeneration lottery.

What to test: Variance across runs at fixed seed; character/style consistency across a set; drift on iterative edits.

11. Commercial licensing and IP safety

Why it matters: You’re selling the printed output. Commercial-use rights, training-data provenance, indemnification, and content/safety filtering are legal exposure, not fine print. A model that’s brilliant but legally murky is a non-starter for a product customers resell.

What to test: Commercial-use terms; indemnity; provenance/transparency; enterprise and data-handling terms; content-filter behavior.

The scoring rubric

Score each model 1–5 on each criterion, then weight by your use case. Suggested weights for the three most common web-to-print contexts:

Criterion	Interactive design tool	High-volume VDP / automation	Merch / POD
1. Vector / SVG output	High	Medium	Medium
2. Resolution / max size	High	High	High
3. Visual quality	High	Medium	High
4. Text rendering	Medium	High	Medium
5. Controllability / editing	High	High	High
6. CMYK-readiness	High	High	High
7. Transparency / cutout	Medium	Low	High
8. Responsiveness	High	Medium (throughput)	High
9. Cost per generation	Medium	High	Medium
10. Consistency	Medium	High	High
11. Licensing / IP	High	High	High

Scoring scale: 1 = unusable for print · 2 = weak · 3 = workable with mitigation · 4 = strong · 5 = best-in-class.

Drop your own numbers into this rubric. The measured scores in the results table later in this guide give you a starting hypothesis to test; the qualitative notes are not a substitute for your own runs on your prompts.

Models by job

New models ship faster than any ranking can keep up with, so this section is organized by job, the way you actually route requests. Models are drawn from the roster currently wired into CE.SDK’s AI plugins.

Job A: Text-to-image (generate a new asset)

Candidates: Recraft V3, Recraft 20B, Seedream V4, Nano Banana, Nano Banana Pro, GPT Image 1, Ideogram V3.

Model	Stand-out for print	Watch-outs	Benchmark
Recraft V3	The print specialist: native vector/SVG generation, strong text rendering, brand-style controls. Often the single most print-relevant text-to-image model.	Verify max raster resolution for large-format jobs.	3.29 / 5
Recraft 20B	Faster, cheaper Recraft tier. Good for interactive iteration before a final Recraft V3 render.	Quality step-down vs. V3.	not yet in the suite
Seedream V4	High visual fidelity and resolution; strong general-purpose hero imagery.	Text rendering and vector are not its strength.	3.80 / 5 (measured on Seedream 4.5)
Nano Banana / Pro	Fast, controllable, strong prompt adherence; Pro for higher quality. Good interactive default.	Confirm commercial-licensing terms for resale.	3.24 / 5 · Pro 2.55 / 5
GPT Image 1	Strong instruction-following and comparatively reliable in-image text; good for layout-aware generation.	Latency and cost on the higher side for interactive use.	3.07 / 5 (measured on GPT Image 1.5)
Ideogram V3	Built around typography and still a credible pick for poster and typographic art, but our measured text column no longer puts it on top.	Ranks twelfth of fifteen on text accuracy in our run.	2.97 / 5

Routing heuristic: logos, icons, and typographic art go to Recraft V3 (vector; the print files use case). Words-in-image goes to whichever flagship currently tops the measured text column, which in our July 2026 run means GPT Image 1.5, FLUX.2 [pro] or Nano Banana Pro rather than a typography specialist (the text-heavy designs use case). Photoreal hero imagery goes to Seedream V4 or Nano Banana Pro (the product ads use case). Fast interactive drafts go to Recraft 20B / Nano Banana.

Job B: Image editing (adapt a customer’s asset)

Candidates: Flux Pro Kontext, Flux Pro Kontext Max, Nano Banana Edit, Nano Banana Pro Edit, Qwen Image Edit, Gemini Flash Edit, Seedream V4 Edit, GPT Image 1, Ideogram V3 Remix.

Model	Stand-out for print	Watch-outs	Benchmark
Flux Pro Kontext / Max	Strong instruction-based editing with high subject preservation: change one thing without wrecking the rest. Good for generative fill/expand and targeted edits.	Max tier costs more; budget per edit.	editing suite planned
Nano Banana Edit / Pro Edit	Fast, controllable edits; good interactive default for object removal and swaps.	Verify edge quality on fine detail.	editing suite planned
Qwen Image Edit	Strong editing with good text handling during edits.	Validate commercial terms.	editing suite planned
Gemini Flash Edit	Low latency: the responsiveness pick for interactive editing.	Quality trade-off vs. heavier models.	editing suite planned
Seedream V4 Edit	High-fidelity edits matching its generation quality.	Heavier; weigh latency.	editing suite planned

The benchmark currently measures text-to-image only; the editing models above join the leaderboard when the frozen suite adds an editing track, so their cells read “editing suite planned” rather than a score.

Routing heuristic: precise “change X, keep everything else” goes to Flux Kontext. Fast interactive edits go to Gemini Flash Edit / Nano Banana Edit. Edits that must preserve in-image text go to Qwen Image Edit.

Job C: Specialized print operations

These aren’t single “models” so much as task pipelines, but they belong in the routing table:

Task	Approach / model	Note
Vectorize (raster → SVG)	Vectorize plugin / Recraft vector	The print-critical one: protects logos and PDF/X vector output.
Background removal	`@imgly/background-removal` (in-browser)	On-device, instant, privacy-preserving. Feeds cutout paths.
Image correction & DPI	Perfectly Clear plugin + DPI validation	Auto-corrects exposure, color, sharpness, and noise; validation flags undersized images. Prefer high-res native generation over upscaling.
Text generation / copy	Claude Sonnet 4.6, GPT-4.1 Nano	Headlines, rewrites, per-recipient VDP copy.
Text-to-speech / sound	ElevenLabs (where relevant)	Less common in print; relevant for multi-channel campaigns.

The quick-reference picks

If you skip the rubric and just want today’s defaults, this is the snapshot we’d start from. Treat it as the hypothesis your own benchmarks confirm or overturn, not a verdict.

Print job	Pick	Why
Logos, icons, line art, anything that must scale	Recraft V3	The only major model with native SVG/vector output. Everything else hands you pixels to trace. Vector survives into PDF/X as real paths and prints sharp at any size.
Legible text inside the image (posters, labels, packaging motifs)	GPT Image 1.5, FLUX.2 [pro], Nano Banana Pro	These three rendered every required string in our suite exactly. The typography specialists did not: Ideogram ranks twelfth of fifteen on the measured column. Still, bake text in only when you must: real type as a separate editor layer beats all of them.
Photoreal hero imagery at print size	Seedream 4.5	Resolution is the print bottleneck, and it is the one criterion where the field genuinely splits. At default parameters Seedream returns 2048px (about 6.8” at 300 DPI); most of the field returns 1024px, which is 3.4” and a bet on the upscaler.
Transparent cut-outs for stickers, die-cuts, merch	GPT Image 1.5, and a background-removal step regardless	It is the only model in our run that reliably emits a real alpha channel. Thirteen of fifteen emit none at all, so the cut-out has to be a pipeline step, not a prompt.
”Change X, keep everything else” edits	Flux Kontext Pro / Max	Best-in-class instruction-following edits with subject preservation, which is critical when the asset is the customer’s product photo and “the AI improved it” means a reprint. The Nano Banana family is the consistency pick for variants of one subject.
Interactive drafts (speed and cost)	Recraft 20B, Gemini Flash Edit, base Nano Banana	In an editor, a 20-second generation is a bounce. Draft cheap and fast, then re-render the final with the heavyweight. The customer never needs to know two models were involved.
Data sovereignty / self-hosting	Qwen-Image / Qwen Image Edit	Apache 2.0 open weights. For European operators and government buyers who won’t send customer uploads to a US API, “we can run it ourselves” beats a quality delta.
VDP copy and headlines	Claude Sonnet 4.6	Cheap enough per recipient at 100k-mailer volume, and strong at length-constrained rewriting: the “fit this text frame” problem.

Three of those rationales carry more weight than the rest.

Vector is the criterion that decides printability. Screen products never need vector, so general models never optimized for it, and Recraft sits almost alone on the one measure that determines whether a logo prints cleanly at size. If you adopt a single routing rule from this guide, make it this one.

Resolution and color are pipeline problems the model can only start. No model outputs CMYK; every one generates sRGB. “CMYK-readiness” is really about how hard a model’s palette clips on conversion, and your ICC and soft-proofing pipeline owns the rest. Same with DPI: pair every generation path with validation and a dedicated upscaler rather than trusting native output. You can swap the model whenever you want; the print-correctness layer around it has to stay put.

Transparency is not a model feature you can shop for. When we asked every model for a transparent background and measured the alpha channel that came back, thirteen of fifteen returned none, and a few painted a checkerboard into the pixels instead. If your product sells stickers, die-cuts, or anything composited onto a garment, budget for background removal and edge cleanup as a permanent pipeline stage. Choosing a different model does not remove that stage.

One roster note: Adobe Firefly isn’t wired into CE.SDK’s plugin roster today, but for licensing-sensitive enterprises its trained-on-licensed-content and indemnification story is the strongest answer to criterion 11. Include it in your own evaluation if criterion 11 dominates your weighting.

A worked decision: which model wins for your product?

The rubric resolves to a different answer per context. Three quick reads:

Interactive design tool (non-designers personalizing templates). Weight responsiveness, controllability, and vector. Likely stack: Recraft 20B / Nano Banana for fast generation, Recraft V3 for final vector, Gemini Flash Edit for interactive edits. The customer iterates fast and the final asset is print-clean. See the in-app UGC use case for how the benchmark weights this context.
High-volume VDP / automation (100k personalized mailers). Weight cost, throughput, consistency, and licensing. Likely stack: a cost-efficient generation model at high concurrency, Claude Sonnet 4.6 for per-recipient copy, seed-locked for repeatability, all run headless. Per-unit cost dominates the decision. See the personalization-at-scale use case for the weighted read.
Merch / POD (user art on physical products). Weight transparency/cutout, resolution, and quality. Likely stack: background removal plus cutout for subjects, upscaling for resolution, Seedream V4 / Nano Banana Pro for generated designs, with hard licensing checks because the output is resold. See the merch and stickers use case, where the transparency gap decides the stack.

In every case, the conclusion is the same: no one model wins all three. The advantage is in being able to route each task to the model that wins it, and to swap models as better ones appear, without re-architecting your editor.

What makes per-job routing practical: native plugins + the AI Gateway

A criteria-driven, route-per-job strategy only works if switching models is cheap. If wiring in a new model means a new API integration, new auth, new key management, new error handling, and new billing plumbing every time, you’ll quietly standardize on whatever you integrated first, and inherit its weaknesses on every job it loses.

CE.SDK is built to avoid that:

Every image model in this guide is natively supported as an AI plugin. Text-to-image and image editing run as drop-in providers inside the editor, not bespoke integrations you build and maintain. Background removal, vectorization, and image correction (Perfectly Clear) ship as their own plugins, and text generation routes through built-in Anthropic and OpenAI providers. Adopting Recraft for vector, Ideogram for text, and Flux Kontext for edits is configuration, not three separate engineering projects.
The AI Gateway gives you one unified surface to every model. Instead of stitching together a dozen provider APIs, key schemes, and rate-limit behaviors, you call models through a single managed layer. That’s what turns “route each task to the best model” from an architecture diagram into a one-line config change, and what lets you swap a model the day a better one ships, without touching your editor or your print pipeline.
Outputs land inside the print pipeline, not beside it. Whatever model produces the asset, it flows into locked templates with bleed, safe-area, and DPI constraints, then through CMYK / spot-color / PDF/X-3 export. The model is interchangeable; print correctness is not. Print-on-demand company The Print Bar and direct-mail platform PostBuddy both built on this pipeline, embedding CE.SDK rather than maintaining their own print editor.

So the “best model” question is no longer a one-time, high-stakes bet on a single provider; it’s an ongoing optimization. The benchmark scores below tell you which model to route each job to today, and the native-plugin and Gateway architecture is what lets you act on that answer and revisit it when the scores change next quarter.

The IMG.LY GenAI benchmark: measured results

IMG.LY’s GenAI benchmark now scores these models on print-representative prompts, and the results are live and explorable at img.ly/ai-benchmarks. The table below is the top-line read for the text-to-image models in this guide: the overall score out of 5, the four print-critical criteria (each out of 5), list price per image, and median (p50) latency. All numbers are from the pilot-0 dataset (July 2026); scores are sorted by overall.

Model	Overall	Text	Color	Transparency	Resolution	$ / image	p50
Nano Banana 2 Lite	3.87 / 5	4.9	2.9	0.0	3.0	$0.02	4.0s
Seedream 4.5	3.80 / 5	4.9	2.9	0.0	5.0	$0.048	12.4s
FLUX.2	3.64 / 5	4.9	2.8	0.0	2.0	$0.013	2.0s
FLUX.2 Turbo	3.61 / 5	4.9	2.7	0.0	2.0	$0.015	2.0s
Gemini 2.5 Flash Image	3.60 / 5	5.0	1.7	0.0	3.0	$0.039	7.4s
Recraft V3	3.29 / 5	4.0	1.9	2.0	3.0	$0.04	7.4s
Nano Banana 2	3.24 / 5	5.0	3.2	0.0	3.0	$0.10	13.2s
FLUX.2 Pro	3.23 / 5	5.0	2.3	0.0	2.0	$0.04	11.5s
Qwen-Image	3.21 / 5	4.8	2.0	0.0	2.0	$0.03	6.9s
GPT Image 1.5	3.07 / 5	5.0	3.4	4.2	3.0	$0.12	34.0s
Ideogram 3.0	2.97 / 5	4.7	2.8	0.0	3.0	$0.06	17.9s
Nano Banana Pro	2.55 / 5	5.0	3.7	0.0	3.0	$0.36	23.3s

Measured on the pilot-0 suite (July 2026): 37 prompts times 3 seeds per model, every image scored 0 to 5 per criterion by an automated vision judge. Two models carry newer versions than the guide names them by: Seedream V4 is measured as Seedream 4.5, and GPT Image 1 as GPT Image 1.5. The suite also covers Seedream 5.0 Lite, Luma Photon, and Stable Diffusion 1.5, which this guide’s roster does not name; their scores are on the rankings page. The overall column is an unweighted blend, so read it alongside the per-use-case weighting, not as a single leaderboard: the cheap fast model tops the raw average while the priciest lands last. Full methodology is at img.ly/ai-benchmarks/methodology, and every model, prompt, and generated image is explorable at img.ly/ai-benchmarks.

A few things the numbers make plain, and that the criteria above predicted. Transparency is close to unsolved: 13 of 15 tested models emit no real alpha channel at all (the transparency finding), and GPT Image 1.5 is the only one that scores meaningfully on it. Brand color is broadly weak: measured with CIEDE2000, no model reliably hits an exact hex (the brand-color finding). Text is the one column where the field mostly clears the bar: on our per-required-string measure several general-purpose flagships render every string in the suite exactly, while the model with the biggest reputation for text, Ideogram, actually lands near the bottom of the column, not the top (the text-reliability finding). The gaps open exactly where print work gets hard. Averaged across the field, the neon sign in Japanese scores 3.91 and the dense packaging label 4.40, against 4.91 for a greeting-card cover. Latin headlines are close to solved; small type and non-Latin scripts are not.

The same numbers, weighted for print

The overall column is the wrong lens for a print product, which is the whole reason the benchmark carries use-case weights. Reweighted for print files (resolution 35%, color 30%, text 15%, transparency 10%), Seedream 4.5 leads at 3.64, ahead of Seedream 5.0 Lite at 3.50 and GPT Image 1.5 at 3.45: native output size at default parameters does most of the work, and the two models that return 2048px start the job at twice the printable width of the 1024px field.

Reweight again for merch and stickers, where alpha carries 35%, and the order breaks completely: GPT Image 1.5 wins at 3.72, nearly a full point ahead of Recraft V3 at 2.78, because it is the only model that reliably returns a real alpha channel. Same measurements, different job, different answer. That inversion is the argument for routing per job rather than standardizing on one provider.

The gated report is coming to subscribers first. AI Image Models for Print Production: The Benchmark Report packages the weighted per-use-case totals, the full test set, and the print-specific analysis into one PDF. Subscribe to get it before it is public.

The criteria and the routing logic above stand on their own. The scored numbers are what the suite adds: a measured, repeatable answer for each cell, refreshed as models change. For why the benchmark is built this way, and what else the run turned up, see Introducing IMG.LY AI Benchmarks.

Bottom line

Pick criteria before you pick models. For web-to-print, the criteria that screen benchmarks ignore (vector output, print resolution, CMYK behavior, text legibility, licensing) are exactly the ones that decide whether a beautiful generation is a sellable print. Score the current models against those criteria for your context, route each job to the model that wins it, and run it all through CE.SDK’s native AI plugins and AI Gateway so re-routing tomorrow, after the next leaderboard reshuffle, is a config change rather than a rebuild.

Companion piece: How to Leverage Generative AI in Web-to-Print covers the use cases, integration patterns, and print-specific pitfalls behind these model choices.

How to Leverage Generative AI in Web-to-Print

Jan — Fri, 26 Jun 2026 00:00:00 GMT

Web-to-print sells self-service: your customer designs a postcard, a flyer, a t-shirt, a packaging label in the browser and orders it without a designer in the loop. For years, though, the self-service stopped at the hard part. Someone still had to supply a usable image. Someone still had to know what 300 DPI meant. When they didn’t, the file landed on a prepress desk to be fixed by hand.

One operator we spoke to put it bluntly: his designers spend roughly 40% of their time fixing customer artwork rather than creating templates. Tom Rowe of The Print Bar described what that looks like at the design step: “Our brand didn’t match our experience. You could see it in our bounce rates.”

Generative AI changes the economics of that hard part. It can turn a prompt into a usable asset, lift a 72-DPI logo to something printable, and strip a background in the browser before a human ever sees the file, removing the specific friction that kept self-service from being self-service in the first place.

It can also go wrong: aimed carelessly, the same models produce beautiful screen images that fall apart on press. This post is about the difference between the two: the use cases worth building, and the print-specific realities you have to design around.

The shift: from “upload a print-ready file” to “describe what you want”

The old web-to-print contract asked the customer to arrive with a finished, technically correct asset. That filter excluded most of the market: the CRM manager, the franchise owner, the small-business buyer who “knows Canva because they make their wedding invitations on it” but has never opened InDesign and never will.

Generative AI moves the burden of production off the customer. Instead of “upload a 300 DPI CMYK image with 3mm bleed,” the ask becomes “type what you want on the card.” The editor, not the customer and not your prepress team, becomes responsible for turning intent into a production-safe file.

That principle runs through everything below: AI is most valuable in web-to-print where it removes a step the customer was never qualified to do. The closer a use case sits to it, the more it returns.

The use cases worth building

1. Generate on-brand imagery from a prompt (text-to-image)

This is the most obvious win. A customer needs a hero image, a background, a seasonal motif, or a product scene, and doesn’t have one. Text-to-image generation lets them describe it and get options in seconds, without a stock-photo license hunt or a design request ticket.

In a print context the point is unblocking the order, not novelty for its own sake. The customer who would have abandoned at “I don’t have a good image” now keeps going. Models like Recraft V3, Seedream V4, Ideogram V3, and the Nano Banana family each have different strengths here (covered in depth in the companion guide, The Best GenAI Models for Web-to-Print), but the integration pattern is the same: generation feeds a placeholder inside a locked template, so the output lands inside your bleed, safe-area, and brand constraints automatically.

Watch out: generated images are usually delivered at screen resolution, often around 1024px. That’s fine for a business card photo, marginal for an A4 flyer, and unusable for large-format. Pair generation with upscaling (below) and size validation before you let it reach export. Native output sizes vary widely by model: the measured native output in the model rankings (pilot-0 dataset, July 2026) shows some models topping out near 1024px while others generate at 2K.

2. Adapt and extend customer-supplied images (image-to-image, generative fill/expand)

Customers rarely arrive with an asset that fits your canvas. It’s the wrong aspect ratio, it has the wrong background, or it’s a portrait crop where you need a landscape banner. Image-to-image editing and generative expand (outpainting) solve the most common case: extending an image to fill bleed or a different SKU’s dimensions instead of stretching or letterboxing it.

This is quietly one of the highest-value print use cases, because aspect-ratio and bleed mismatches are a top source of prepress rework. Generative fill also handles object removal (“take the coffee cup off the desk”), background swaps, and clean-up: the edits a non-designer can’t do in any tool they own.

Watch out: outpainting invents pixels. On a brand asset or a product photo, “invented” can mean “wrong.” Keep generative expand to backgrounds and ambient areas. Never let it reconstruct a logo, a face, or a product the customer is actually selling.

3. Background removal and cutouts for mockups and merch

Background removal is the workhorse, and it has to be a pipeline step because the models won’t do it for you: our transparency finding shows 13 of 15 tested models emit no real alpha channel at all. For merch and print-on-demand it’s the difference between a customer’s snapshot and a clean subject that drops onto a t-shirt, mug, or sticker. IMG.LY runs this in the browser (via the open-source @imgly/background-removal package), which matters for two reasons: it’s instant enough for an interactive editor, and the customer’s image never has to leave the device.

Paired with the cutout plugin, background removal also feeds the literal cut lines for die-cut stickers, labels, and packaging, turning a removed background into a production path.

Watch out: browser-based removal is excellent on clear subjects and struggles on hair, glass, and fine fringes. For a print product that will be inspected up close, expose a manual refine step rather than trusting the mask blindly.

4. Vectorize raster art into print-scalable graphics

This is the use case print people care about most and screen-first builders forget. A customer uploads a logo as a small, jagged PNG. Printed at size, it’s mush. Vectorization (AI raster-to-SVG) traces it into clean, resolution-independent paths that scale to any output size and reproduce crisply on press.

Vectorize is also how you get logos and simple graphics into a state your PDF/X pipeline can preserve as vector, instead of rasterizing everything and throwing away the scalability you vectorized for. Recraft V3 is notable here because it can generate natively vector output rather than only tracing after the fact.

Watch out: vectorization is great for logos, icons, and flat art. It’s the wrong tool for a photograph. Detect the asset type and route photos to upscaling, not tracing.

5. Resolution and image quality: upscaling, correction, and DPI repair

The single most common web-to-print failure is a low-resolution image: the 72-DPI photo that looks fine on screen and prints as a blurry mess. Two different AI operations fix it, and it pays to keep them straight, because they solve different problems.

Upscaling (super-resolution) raises the actual pixel count, reconstructing detail to lift a small image toward a printable size. It’s a generative step you route to a dedicated model, the same way you route a text-to-image prompt; the strong upscalers can take a 1024px generation to 4K. Reach for it when the source is simply too small for its placed size.

Image correction is the other half, and it ships in CE.SDK today as the Perfectly Clear plugin. It auto-corrects exposure, contrast, color, tint, sharpness, and noise in a single pass. It won’t add pixels, but it gets the most printable result out of the pixels you have, which is what a dim, soft phone photo usually needs more than raw resolution.

Tie both to your DPI validation. When the editor detects an image below your minimum threshold for its placed size, offer to correct or upscale it instead of just blocking export. You convert a dead end into a one-click fix, the kind of “they don’t need to know what DPI is” experience operators are chasing.

Watch out: upscaling fabricates detail. It rescues marginal images; it can’t conjure a sharp 4-megapixel product shot from a thumbnail, and correction can’t add resolution at all. Set honest thresholds, and still warn when the source is hopeless.

6. Copy and variable text generation (text + VDP at scale)

Generative text earns its place in two spots. First, as a writing aid inside the editor: headline options, a tagline, a “make it shorter / more formal / fit this space” rewrite for the non-writer staring at an empty text box. Second, and more powerfully for print, in Variable Data Printing: generating or localizing per-recipient copy across thousands of personalized postcards, mailers, or labels from a single template.

VDP is the highest-leverage place to use generative text: generate the variants, merge them into print-ready files in headless mode, and run the batch.

Watch out: generated copy needs guardrails. Length limits so it doesn’t overflow the text frame, tone and brand constraints, and a human-review gate for anything legally sensitive: pricing, claims, regulated industries.

7. Template adaptation and auto-resize across SKUs

A print catalog is the same design across many sizes and substrates: the same campaign as a postcard, a flyer, a poster, a social tile. AI-assisted resize and re-layout adapt one master design to every SKU’s dimensions, reflowing content intelligently instead of scaling it blindly. Combined with locked templates, you expand SKUs without re-authoring each one.

Watch out: reflow decisions still need brand rules. Lock what must stay fixed (logo size, safe area, mandatory legal text) so the AI rearranges within your constraints, not over them.

8. Guided prepress correction

Instead of rejecting a bad file after the fact, use AI to catch and fix problems inside the editor: flag the low-res image and offer to upscale it, detect text outside the safe area and nudge it in, notice a near-white “white” that won’t print and correct it. The same problems your prepress team used to fix by hand get caught here, before the order is placed.

Watch out: auto-correction must be transparent and reversible. Show the customer what changed and let them undo it. Silent “helpful” edits to someone’s artwork erode trust fast.

9. Localization and market variants

For franchise networks and multi-market brands, AI translation plus regeneration of localized imagery and copy turns one approved master into 12 market versions, each print-ready and on-brand, without a translation agency or 12 design tickets. A 200-location franchise can ship 200 localized flyers with the same brand guideline enforced by the editor on every one.

Watch out: machine translation in regulated or legal copy needs human sign-off, and text expansion (German runs roughly 30% longer than English) will break tight layouts unless your template frames can flex.

10. Realistic product previews

Generative and 3D-assisted mockups show the design on the actual product: the shirt, the mug, the folded brochure, the box, not an abstract artboard. Operators consistently tie this to higher conversion and fewer “will this look right?” support tickets, because the preview matches what arrives on the doorstep.

Watch out: a preview that’s prettier than the print sets up a disappointed customer and a reprint. Calibrate mockups to real output, including substrate color and finish.

The print-specific realities

Most generative models are built for screens. Print adds requirements the web doesn’t have, and the ones below are where projects most often break.

Resolution and DPI

Generated images typically arrive at around 1024px. At 300 DPI that’s about a 3.4-inch image: fine for a business card, not for a poster. Always validate output resolution against placed size, and route undersized assets through upscaling before export. Treat raw generation resolution as a starting point, not a deliverable. Native output size is one of the axes we measure: see the measured native output per model to know which models start closer to print size and which need the most upscaling.

Color: sRGB in, CMYK out

Models generate in RGB. Print is CMYK, plus spot colors. The vivid blues, greens, and oranges that look great on screen sit outside the CMYK gamut and will shift on press. And even inside RGB, hitting an exact brand hex is hard: our brand-color finding measures color accuracy with CIEDE2000 and no model reliably lands the color in your brand book, which is one more reason correction belongs on the canvas. Your pipeline, not the model, owns this: convert with the right ICC profile, validate against the print provider’s color requirements, and preserve named spot colors through to the PDF/X. Show the customer a soft-proof so the shift isn’t a surprise on delivery.

Vector vs. raster (and the “everything got rastered” trap)

Most models output raster. For logos, type, and line art that need to scale and print crisply, raster is the wrong format, and a PDF that rasterizes all text and vectors is, in one operator’s words, “death for the printer.” Use vector-native generation (e.g. Recraft) and vectorization for the elements that need it, and make sure your export keeps vector and text as vector with CMYK values preserved (PDF/X-3), not flattened to pixels.

Text rendering inside images

Generative models have a reputation for garbling text inside images, and our text-reliability finding shows the top of the field has largely fixed it: a handful of flagships now render every required string in our suite exactly. The catch for print is that reputation and measurement no longer line up. Ideogram, the model best known for typography, ranks twelfth of fifteen on the measured column; several general-purpose flagships beat it. Reach for a specialist on reputation and you can end up with worse type than the default model you already route to. Put every model on the same wordmark prompt to see the spread. For anything that must be readable in print, don’t bake text into a generated image. Generate the imagery, then set real, editable, vectorizable type as a separate layer in the editor. If you must bake text in, pick the model from the measured column rather than from the marketing; the companion guide has the current ranking.

Brand safety and guardrails

Let customers generate anything and you lose brand control. Box generation inside locked templates instead: it fills a defined placeholder, within fixed margins, alongside a logo and palette the customer can’t move. A franchisee can change the headline and the photo; the logo size, position, and safe area stay locked. AI expands what the customer can create without expanding what they can break.

Commercial and licensing rights

A web-to-print product is sold. That makes the IP status of generated assets a question you have to answer before you ship. Does your model provider grant commercial-use rights to outputs? Are there indemnities? Does training-data provenance create exposure for a customer reselling the printed product? Pick models and providers whose commercial terms you’ve actually read, and surface usage terms where they matter.

Cost and latency (the unit economics)

Every generation costs money and time. In an interactive editor, a 30-second generation is a broken experience. In a high-volume VDP run, a few cents per asset multiplied by 100,000 recipients is a real line item. The spread is not small: the measured $/image and p50 latency in the model rankings run from a fraction of a cent to tens of cents, and from a couple of seconds to over thirty. Match the model to the job: a fast, cheap model for interactive iteration, a higher-quality (slower, pricier) model for the final render, and use the compare tool to weigh the trade-off head-to-head. Cache aggressively. The companion guide scores models on that trade-off.

Data privacy and sovereignty

Customer-uploaded images can be sensitive: faces, IDs, confidential product designs. Browser-based processing (background removal, some editing) keeps data on-device. API-based generation sends it to a third party, which has GDPR and data-residency implications, especially for European customers and government buyers who “prioritize a European vendor for data sovereignty.” Be explicit about what runs locally vs. in the cloud, and choose providers accordingly.

Hallucination and consistency

Generative output is non-deterministic. The same prompt yields different results, and details drift. Our consistency finding measures the worst of it: no model holds a described character across scenes, so series work needs identity as a reusable asset, not a regeneration lottery. For brand assets, product likenesses, and anything a customer will compare against reality, constrain hard (reference images, seeds, image-to-image rather than free generation) and keep a human gate on the outputs that matter.

How IMG.LY approaches it

All of these constraints point the same way: AI is a step inside a print pipeline, not a feature bolted onto an editor. A few principles shape how CE.SDK handles it:

Model-agnostic by design. The AI plugins connect to any third-party model or API: bring your own model. You’re not locked to one provider’s quality, price, or licensing terms. You route each task to the model that wins it, and swap models as better ones ship every few weeks.
The AI Gateway provides managed access to models for editors, so you’re not stitching together a dozen API integrations and key-management schemes yourself.
Generation lands inside the print pipeline. Outputs flow into locked templates with bleed, safe areas, DPI thresholds, and brand constraints already enforced, then through a CMYK / spot-color / PDF/X-3 export that keeps vectors and text as vectors. The model never gets to bypass print correctness.
Drop-in generative features. Background removal, generative fill, image-to-image, vectorize, and text-to-image are available as plugins, so you adopt the use cases above incrementally rather than rebuilding your editor.
Headless mode runs the same generation and export server-side for batch and variable-data runs.

The result is the experience operators actually want to sell: the customer describes what they want, the AI does the part they were never qualified to do, and the file that reaches the press is print-correct because the pipeline enforced it, not because the customer got lucky. For the product-level view of this stack, see AI for Print on Demand.

What customers are doing with it

High-volume print and direct-mail platforms are already building on this foundation. IMG.LY powers print and personalization workflows for operators including Postbuddy (personalized direct mail), Swiss Post, Digitas, and HP, alongside hundreds of smaller print, merch, and franchise platforms.

The pattern that recurs in those conversations is narrow and specific: AI removed a bottleneck. The prepress team that fixed 40% of incoming files now lets the editor catch problems upstream. The customers who bounced at “I don’t have a good image” finish the order. The franchise network that needed a designer for every local variant generates them inside guardrails. It comes back to the point from the top: AI pays off where it deletes a step the customer couldn’t do and the operator didn’t want to.

Where to start

You don’t need all ten use cases on day one. The highest-ROI starting points, in order:

Background removal: instant, in-browser, immediately useful for any merch or photo product.
Image correction and DPI validation: catches and cleans up weak uploads before they ever reach the printer.
Generative expand / fill: kills aspect-ratio and bleed rework.
Text-to-image into locked placeholders: unblocks the “I don’t have an image” abandoner.
Vectorize: rescues logos and protects your PDF/X output.

Each one removes a documented source of friction or rework. Add them inside locked templates and a real CMYK/PDF/X pipeline, and self-service web-to-print finally works the way it was always sold: the customer really can do it themselves.

Next: which models to actually use. See the companion guide, The Best GenAI Models for Web-to-Print: A Buyer’s Guide, for a criteria-based comparison across output quality, vector/SVG support, text rendering, color, responsiveness, and cost. For the measured evidence behind these pitfalls, see the IMG.LY GenAI Benchmarks: every model on the same prompts, scored per criterion, with the method and headline results written up in Introducing IMG.LY AI Benchmarks.

AI Design Agents and Creative Automation: How to Ship a Full Campaign Without a Designer

Klaudia — Wed, 01 Apr 2026 10:14:03 GMT

You have your files, hooks, Google Ads data but still can’t actually launch a campaign because you have to go to a designer for the final assets.

That gap is exactly where campaign momentum dies. Not at the strategy stage or in copy but at the last mile, when everything is ready except the thing people will actually see.

Luckily, it no longer has to.

The AI Marketing Stack Has a Design-Shaped Hole in It

Most marketing workflows have been quietly transformed over the last two years. Copy generation, audience segmentation, keyword research, performance analysis - all of it runs faster now, with less manual input. A solo marketer can do what used to require a team. Except for one part.

Design hasn’t moved. The workflow is still: write a brief, hand it to a designer, wait, review, revise, wait again. Then do the same thing in three more formats because the 1:1 you approved doesn’t fit Stories, Display, or LinkedIn. That loop can take days. And it doesn’t matter how good your AI-generated copy is if it’s sitting in a doc waiting for someone to have bandwidth.

This isn’t a resourcing problem. Hiring more designers doesn’t fix the structural issue; it just adds capacity to a fundamentally slow process. The problem is that the tools built for design weren’t designed for the workflow a modern marketer actually runs. They expect a designer at the keyboard. They don’t expect a marketer with a campaign brief and a conversation window.

The result: campaigns that are otherwise fully automated still stall before they ship. The bottleneck moved from copy to creative. And most teams haven’t noticed yet, because design delays feel normal. They’ve always been there.

What an AI Design Agent Actually Changes

An AI design agent is an autonomous AI system, not a prompt-response tool. It plans, reasons, and executes design tasks independently. Given a goal, it breaks that goal into steps, uses the tools available to it, retains context across the session, and can self-correct when the output isn’t right. That’s the category: a system that drives the workflow rather than waiting for instruction at each step.

Most design agents deliver speed. The handoff that used to take days can happen in minutes. But the output is still a deliverable: a file you receive, use as-is, or move somewhere else. Most operate inside existing tools or generate assets you work with elsewhere. The speed is real, the editability usually isn’t. If something needs to change, you’re back to prompting from scratch.

CoDesign sits differently. It doesn’t hand you a finished asset. It gives you a working starting point on a real canvas. Layers, text boxes, placeholders - real assets you can continue to work with, in the same conversation, without leaving the session.

Brand consistency gets handled at the foundation. Load your brand kit once, including colors, fonts, logo, and layout rules, and every output that session respects those constraints. You’re not eyeballing hex codes or hoping the font looks right. The rules are applied from the start, not checked at the end.

Multi-format adaptation is where the time savings become concrete. A campaign that runs across Instagram, Google Display, LinkedIn, and print doesn’t produce four separate briefs and four separate rounds of designer revisions. You describe the formats you need, and the agent adapts the work. The campaign stays consistent across all of them.

The biggest shift isn’t speed, though that’s real. It’s that you stay in creative control throughout. There’s no handoff moment where you lose the thread and have to re-explain the brief to someone else. The context lives in the conversation.

How to Run a Campaign Production Session with CoDesign

This is the actual sequence. Walk through it once and the workflow becomes repeatable.

1.Start with the brief. Open CoDesign and describe the campaign in plain language. Audience, objective, platform, tone, any constraints. The more specific you are here, the less iteration you’ll need later. Treat it like briefing a senior designer who hasn’t worked with your brand before.

2.Generate copy variations before you open the canvas. Use whichever AI writing tool you already work with (ChatGPT, Claude, or whatever is in your stack) to produce multiple copy directions: headline hooks, body copy, CTAs. Get four or five versions per element. Copy and design are separate steps that feed into each other. Having real options ready before you start the design session means CoDesign has something specific to work with, not a blank brief waiting to be interpreted.

3.Feed the brief to the AI design companion. With copy variations ready, ask CoDesign to generate initial ad designs. Describe the format, the hierarchy you want, any layout preferences. You’ll get structured, editable designs on the canvas. Not a rendered image, but a working starting point with real layers.

4.Apply your brand kit. If you haven’t already, load your brand assets: logo, color palette, type system, approved imagery. The agent applies these rules to the designs. Every output from this point respects your brand standards automatically.

5.Adapt across formats. Tell the agent which formats you need. The social variant, the display variant, the vertical for Stories, the square for feed. Watch the layouts adapt to each context, maintaining the campaign idea and brand consistency across dimensions. If the hierarchy needs adjusting for a specific format, describe what’s not working and the agent fixes it in conversation.

6.Refine in conversation. This is where the canvas-based approach earns its value. You’re not generating new versions from scratch. You’re iterating on what’s already there, in the same session. “Move the logo to the bottom right. Try the headline in the lighter weight. Swap this layout for something with more white space.” Each exchange builds on the last, so the conversation stays grounded in what’s on the canvas rather than starting over from a new prompt.

7.Export and ship. When the designs are approved, export in the formats your media plan requires. The session context lives with the file, so if something needs to change post-launch, you’re not starting from zero.

One honest note: the quality of the output is proportional to the quality of the brief. Vague prompts produce generic starting points. Teams that invest two minutes in a specific, structured brief consistently get more usable first outputs than teams that describe the campaign in one sentence and expect the agent to fill in the gaps.

This Is What Closing the Loop Actually Looks Like

Design agents don’t replace designers. They remove the bottleneck that sits between strategy and execution.

The marketer who used to wait three days for ad assets can now produce a full campaign set in a single session. The designer who spent half their week on format resizes and small-copy tweaks can spend that time on the work that genuinely requires their judgment. That means brand-defining creative, campaign concepts, and anything where taste and experience are the actual input.

It was never about willingness to collaborate. The tools just didn’t allow for anything else. Every design change, no matter how small, had to go through a handoff. A headline adjustment on a banner resize does not need a creative director. A resize from 1:1 to 9:16 does not need a brief, a Slack message, and a 48-hour turnaround.

Thanks to design agents conversation shifts. It moves from “can you make this” to “how should this look.” And that’s a way more interesting conversation.

Interested in trying IMG.LY CoDesign? Reach out to our team.

What Is an AI Design Agent?

Klaudia — Mon, 30 Mar 2026 04:52:01 GMT

The phrase “design agent” is being used in two completely different ways right now, and the confusion is worth clearing up before either meaning becomes the default. Search for it today and you’ll mostly find AI design agencies - services firms that use AI in their process. That’s a reasonable business, but it has nothing to do with what this article is about.

The second meaning, a design agent as a category of AI tool, is the one that matters for product builders, developers, and creative teams trying to understand where AI-assisted design is actually going. That’s what we’re defining here: what it is, how it works, how it differs from tools you already know, and why the distinction matters if you’re building or evaluating creative software in 2026.

First, What an AI Design Agent Is Not

An AI design agency. This is a services firm that uses AI tools in its creative process. It’s a completely different category - a service, not a tool - and it’s the most common result you’ll find when you search “design agent” right now.

An AI image generator. Midjourney, DALL-E, and similar tools produce images from text prompts. They generate a single output from a single instruction.

An AI feature inside a design tool. Figma’s AI suggestions, auto-layout assistance, and background removal are AI features. They augment a manual workflow making specific tasks faster.

A design automation pipeline. Server-side systems that batch-generate assets from templates are automation, not agency. They’re fast and scalable, but they don’t converse, can’t interpret an ambiguous brief, and don’t refine their output in response to feedback. The confusion here is understandable: modern automation systems use AI models internally and can produce polished, varied-looking results that are easy to mistake for something more intelligent. But give one an ambiguous brief and it either fails or produces something technically correct that misses the point entirely. Give a design agent the same brief and it asks a question. That distinction - executing a fixed process versus reasoning about intent - is what separates the two categories.

The distinction matters because “design agent” is becoming a meaningful category term, and what it actually describes is different enough from all of the above that collapsing the distinctions creates real confusion, both for people evaluating tools and for people building products.

AI Design Agent - a Working Definition

There is no settled definition of an AI design agent yet, the term is still early. Based on what we see in the tools that genuinely deliver on the promise, three properties distinguish a real design agent from something that merely resembles one:

1. Autonomy. The agent takes multi-step actions without requiring a human instruction at each step. Given “create a five-page product catalog in a Scandinavian minimal style,” it works out the layout, typography, and image placement independently and produces the result.

2. Conversational interface. The agent communicates in natural language. It can ask clarifying questions, explain what it’s done, and accept follow-up instructions. The interaction feels like briefing a designer, not operating software.

3. Refinement loop. The agent takes feedback across multiple conversation turns: “make the typography lighter,” “apply the brand’s warm color palette”, and updates the design accordingly. This iterative loop is what separates a design agent from a one-shot generation tool.

A tool that has all three of these properties is a design agent. A tool that has one or two is something else; probably useful, but in a different category. IMG.LY CoDesign is built against exactly this bar: it takes a brief, works through the layout on its own, and refines the result in conversation.

How an AI Design Agent Works in Practice

An abstract definition only takes you so far. Here’s what a fully implemented design agent workflow actually looks like, drawn from a real demo of IMG.LY CoDesign. CoDesign runs as a design MCP server, which means the same agent can also work from Claude, Cursor, or any other MCP client, not just a built-in chat panel.

Step 1. The user opens CoDesign. The agent interface - a chat panel - sits alongside the canvas. They’re in the same window.

Step 2. The user types a brief: “Here’s a CSV of five products from our furniture brand. Generate one landscape catalog page per product — two-column grid, full-bleed photo left, typography-led layout right. Clean, minimal. Think Hay, Muuto, Frama.”

Step 3. The agent processes the brief, the data, and any brand context it has access to. It generates a five-page catalog in the editor: product names, descriptions, prices, photo placeholders, layout structure consistent across all five pages.

Step 4. The user reviews the output and follows up in the chat: “Nice. Pre-fill the photo placeholders with black-and-white product photography, soft contrast.”

Step 5. The agent updates the design. The user manually adjusts one headline, corrects a price, and exports.

The full workflow, from brief to print-ready output, took minutes rather than hours. The user never left the tool. The agent handled all structural and stylistic decisions. The human handled final judgment and small corrections.

But generating layouts from a brief is only part of what a design agent can do. In the same chat interface, the agent can build functional decision-making tools directly inside the editor. Ask it to create a Color Themes panel, and it produces a working component: five named presets, each one applying a complete color theme across the entire design in a single click. The user doesn’t switch to a settings screen or manually update individual elements. The agent has built the control they need, right where they need it, as part of the same conversation. That’s a meaningfully different capability: not just producing a designed output, but constructing the tools that let the user make better decisions about that output as they finalize it.

This table is meant as a fair comparison, each column represents how these tool categories actually behave today:

Capability	AI Image Generator	AI Design Features	Design Agent
Takes a brief	Prompt only	No	Yes
Produces editable layouts	No	Partial	Depends on a tool
Multi-step autonomy	No	No	Yes
Conversational refinement	No	Limited	Yes
Brand / context awareness	No	Partial	Yes
Iterative across a session	No	No	Yes

The pattern here isn’t that design agents are better at everything - it’s that they operate at a different level of the workflow. Image generators and AI design features are task-level tools. A design agent is a workflow-level tool.

There’s one adjacent term worth separating out too: vibe design, the workflow where you describe what you want instead of building it manually. A design agent is one way to run a vibe design workflow, but the tools behind it differ a lot in what they hand back. We’ve compared the AI design tools built around this workflow if you want to see how they stack up.

Agentic Design vs. AI Design Agents: What’s the Difference?

“Agentic design” and “AI design agent” look interchangeable, and they’re starting to get used that way, but they describe different things.

Agentic design is a practice: the set of patterns and decisions involved in building systems where an AI acts with some autonomy. How much the agent does before checking in, when it asks a clarifying question, how it surfaces what it changed - these are agentic design questions, and they apply to coding agents and support agents just as much as to creative tools.

An AI design agent is a product: a tool that applies that practice to design work specifically. The autonomy slider in the next section is a good example of the overlap. Deciding where that slider sits is agentic design; the tool you move it in is a design agent. If you’re evaluating tools, you’re shopping for a design agent. If you’re building one, you’re doing agentic design.

Creative Agents: The Wider Category

Zoom out one level and design agents sit inside a wider family: creative agents. A creative agent is any AI agent that produces creative output autonomously - copy, video, music, imagery, or design. Design agents are the branch of that family that works in layouts, typography, and brand systems.

The defining properties carry across the family: autonomy over multi-step work, a conversational interface, and a refinement loop. What changes is the medium and the tooling around it - an agent cutting video needs a timeline the way a design agent needs a canvas. Keeping the levels straight helps when you’re mapping the space: the creative agent is the category, the design agent is its design-specific branch, and the products above are competing to define what that branch looks like.

The Autonomy Slider: Why Human Control Still Matters

One of the most important design decisions when building or evaluating a design agent is how much autonomy the AI exercises by default.

Full autonomy is fast. The agent makes all decisions and presents a finished output. But it can produce results that don’t match the user’s actual intent, especially when the brief is ambiguous or the stakes are high.

Minimal autonomy is safe. The agent only suggests, the human decides everything. But at that point, you’ve lost much of the value of having an agent in the first place.

The best implementations give users what you might call an autonomy slider: the ability to let the agent run with full independence on some tasks, take targeted direction on others, and step aside entirely when the user wants to edit manually. The right level of autonomy depends on the task and the user’s confidence, not a fixed setting applied uniformly across every interaction.

For any design agent, this means the interface needs to let the user reach in and adjust manually, mid-workflow, without losing what the agent has already done. The agent and the editor have to exist in the same environment. Separate windows with an export step between them break the loop entirely.

Where AI Design Agents Create the Most Value

Not every creative workflow benefits equally from an AI agent. The scenarios where design agents consistently create the most value tend to fall into three categories:

High-volume, high-variation creative production. Marketing teams producing dozens of ad variants, e-commerce teams generating product imagery at scale, publishers creating template-based content in bulk. A team can brief the agent once: brand colors, copy, format specs, and get 40 properly formatted variants back in the time it would have taken to manually produce three, each one consistent with the last.

Non-designer users who need professional results. Not everyone who needs to produce a designed output is a designer. Marketers, retailers, operations teams, small business owners - they know what they want but don’t have the tools or time to build it manually. A design agent gives them a way in that doesn’t require either.

Expert designers who want to explore faster. Experienced designers using the agent to generate starting points, explore multiple directions quickly, or offload the time-consuming production work while retaining full control over the final result. A designer can spend 20 minutes reviewing six distinct layout directions the agent generated from a single brief, rather than a full afternoon building each one from scratch.

These aren’t exhaustive categories, but they represent the use cases where the workflow-level shift actually changes what’s possible, not just what’s faster.

A Different Kind of Design Tool

A design agent makes creative work faster and more accessible. For designers who want to explore more directions in less time, and for non-designers who have always known what they wanted but lacked the tools to build it.

That’s what IMG.LY CoDesign is built around: editable designs straight from your agent, inside a fully featured design editor. The local MCP server is free to install and run, or get in touch if you want to talk it through with the team.

Frequently Asked Questions

What is an AI design agent?

An AI design agent is an AI tool that takes a design brief in natural language, produces an editable design on its own, and refines it through conversation. Three properties define the category: multi-step autonomy, a conversational interface, and a refinement loop. A tool that has all three is a design agent; a tool with one or two belongs to a different category.

How is an AI design agent different from an image generator?

An image generator produces a single flat image from a single prompt. A design agent works at the workflow level: it interprets a brief, makes multi-step layout and typography decisions, produces structured output you can edit element by element, and takes feedback across a session.

Do AI design agents replace designers?

No. The most effective setups keep a human in charge of direction and final judgment. Non-designers use agents to reach professional results they couldn’t build manually; experienced designers use them to explore more directions in less time while keeping full control over the result.

Are AI design agents free?

It varies by tool. Google Stitch is free during its beta with monthly generation limits, Figma Make requires AI credits, and IMG.LY CoDesign’s local design MCP server is free to install and run, with no account needed to start.

Best AI Design Tools Compared: 6 Vibe Design Platforms Tested

Klaudia — Sun, 29 Mar 2026 03:51:56 GMT

The best AI design tools no longer just generate images: they take a brief and hand back a finished design. The newest of them are AI design agents, tools you direct in conversation rather than operate click by click. The workflow has a name, vibe design: vibe coding gave non-technical people the ability to build software by describing what they wanted, and the same shift is now happening in design. For the first time, a marketer, a retailer, or a small business owner can create a professional design exactly to their specification just by explaining it to an AI. No design skills required. No external tool. No waiting for a designer to become available.

This article compares six tools leading that shift - what they do well, where they differ, and how to choose between them.

Tools Built for Creators

These tools represent the current state of standalone vibe design software. Each is genuinely good at what it does. The differences come down to what kind of work they’re built for.

Google Stitch

Google Stitch is an AI-native design canvas from Google Labs, launched for public beta in early 2026. It accepts text, voice, images, and code as input and produces high-fidelity UI designs and interactive prototypes on an infinite canvas. Its design agent can reason across an entire project rather than just the current frame - a meaningful step beyond tools that only operate on one screen at a time.

Stitch introduces a DESIGN.md file format for exporting and importing design rules across tools, and it connects to Cursor, Claude Code, and Gemini CLI via an MCP server. It’s available free during beta with monthly generation limits, and it’s clearly aimed at UI/UX designers and developers building product interfaces - not marketing or graphic design work.

Figma Make

Figma Make takes natural language prompts or existing Figma designs and produces working prototypes and web apps - functional code, not just mockups, with optional Supabase backend integration. Because it lives inside Figma, it has access to your existing design libraries, component systems, and tokens from the start.

It’s best understood as a rapid ideation and prototyping tool. It generates interactive, working outputs fast, but the code benefits from developer review before going to production. It’s distinct from Figma Sites, which handles actual publishing. AI credits are required for usage.

Lovart

Lovart is a standalone AI design agent founded by a former ByteDance senior product director. Its scope goes well beyond graphic asset creation: from a single prompt, it can generate brand identity systems, UI mockups, video content, packaging designs, and full marketing campaigns. It uses a proprietary MCoT (Mind Chain of Thought) reasoning engine designed to mimic how a creative director thinks - analyzing business context, target audience, and brand requirements, not just aesthetic style.

Its infinite canvas continuously analyzes all assets present to maintain visual consistency across an entire project, and outputs are compatible with Figma, Photoshop, and After Effects. Lovart is positioned for marketing creatives, solo designers, and small creative teams who need to produce full campaigns without a large agency.

Adobe Firefly Boards

Adobe Firefly Boards is Adobe’s AI-first collaborative ideation canvas, built for creative professionals who need to move quickly from inspiration to concept. It lives inside the Adobe Firefly web app and mobile apps (iOS and Android), and syncs with Creative Cloud so work can move from moodboard to production without switching platforms.

The core of Boards is an infinite multimedia canvas where you bring in text prompts, reference images, video clips, and Adobe Stock assets, then generate across all of them at once. What makes it unusual is the model breadth: Boards gives you access to Adobe’s own Firefly models alongside partner models from Runway, Pika, Luma AI, Google Veo, Black Forest Labs, and others, all inside the same workspace. Outputs are images, video clips, text overlays, and assembled mood boards or storyboards, and every generated asset carries Content Credentials that automatically record which AI model produced it.

Canva Magic Studio

Canva Magic Studio is a suite of more than 15 AI-powered tools embedded directly into the Canva editor on web, mobile, and desktop. It’s not a separate app. If you’re already a Canva user, you’re already inside Magic Studio.

The suite covers writing, image generation, video creation, format switching, and multi-language translation, all accepting text prompts, uploaded media, documents, and existing canvas elements as input. Brand Kit integration is what distinguishes it from more isolated AI generation tools: when you generate a design through Magic Design, it pulls from your saved Brand Kit (logo, colors, fonts) rather than producing something generically styled. That keeps AI-generated output on-brand without manual cleanup.

The trade-off is structural. Canva outputs are flat exports within a consumer-grade editor, not structured design objects you can manipulate at the element level. Generations are fast and on-brand, but the editing ceiling is Canva’s editor. For users who need granular creative control or complex asset structures, that’s a real constraint, not just a preference.

IMG.LY CoDesign

IMG.LY CoDesign is AI-powered design companion built for creators and teams producing marketing materials, branded content, social assets, and multi-format campaigns. What sets it apart from other standalone tools is how it treats AI output: every element CoDesign generates is a structured, editable design object on the canvas - not a flat image to export and rebuild elsewhere. Move a headline, swap a font, resize a block, change a color - directly in the same tool that generated it.

The agent and the editor aren’t two modes. They’re the same tool. You can prompt a direction, refine it in conversation, and take over the canvas directly at any point - without switching tools, exporting, or starting over. CoDesign also goes beyond the canvas: ask it to build a brand panel, a form-based template, or a custom configurator, and it builds that too - directly inside the editor, from the same prompt interface.

CoDesign also ships as a free local MCP server for design, so the same agent works from Claude, Cursor, or any other MCP client, not just its own editor.

Vibe Design Tools at a Glance

	Google Stitch	Figma Make	Lovart	Adobe Firefly Boards	Canva Magic Studio	IMG.LY CoDesign
Primary use	UI/UX design from prompts	Prompt-to-prototype / app	Full campaign and brand creation	AI-first concepting and moodboarding	AI design generation inside Canva editor	AI-assisted design for creators and teams
Where it lives	Google (standalone web app)	Inside Figma	Lovart (standalone)	Adobe Firefly web + mobile; syncs with Creative Cloud	Inside Canva (web, mobile, desktop)	IMG.LY
Input types	Text, voice, images, code	Text, existing Figma designs	Text, images, brand briefs	Text, images, video, Adobe Stock assets	Text, images, video, documents, existing design elements	Text, images, CSV, brand kits
Output type	UI designs and prototypes	Interactive prototypes and web apps	Brand assets, campaigns, video, packaging	Images, video, text overlays, mood boards, storyboards	Images, video, presentations, styled text, format-converted designs	Editable multi-page designs, videos, animations
Output format	Flat / exportable	Code / exportable	Flat / exportable	Flat / exportable; Content Credentials attached	Flat / exportable within Canva editor	Structured editable objects on canvas
Brand context	Manual per session	Via Figma design system	Canvas-aware within session	Enterprise Custom Models; personal library import	Brand Kit applied automatically at generation	Loaded at session start
In-chat UI generation	No	No	No	No	No	Yes
Multi-model AI	No	No	No	Yes (Runway, Pika, Luma, Google Veo, BFL, and more)	No (Dream Lab uses Leonardo.ai Phoenix)	No
Best for	Product designers, developers	Product teams in Figma ecosystem	Marketing creatives, solo designers	Creative professionals doing concepting and ideation	Non-designers and content teams needing fast, on-brand output	Creators and teams needing editable AI-generated design

For a detailed breakdown of how Canva Magic Studio compares to CoDesign specifically, see Canva Magic Studio vs. CoDesign. If you’re evaluating from the developer side, CoDesign is also the one that doubles as an AI design tool for developers: it registers as a local MCP server, so design generation becomes a capability of whatever agent stack you already run.

How to Choose

The criteria here aren’t about feature counts. They’re about the kind of work you’re doing.

Choose Google Stitch if you’re a product designer or developer building UI and want an AI agent that reasons across your whole project, not just individual screens. It’s the most technically connected of the six (MCP server, code input, DESIGN.md export), and it’s clearly built for product interface work.

Choose Figma Make if you’re already in the Figma ecosystem and want to go from prompt to working prototype fast. It inherits your design system automatically, which removes a lot of setup friction, and it outputs functional code rather than static designs.

Choose Lovart if you need to produce full creative campaigns (brand identity, packaging, video, marketing assets) from a single prompt. Its MCoT reasoning engine and canvas-level consistency analysis make it particularly strong for high-volume marketing creative work.

Choose Adobe Firefly Boards if you’re a creative professional working inside the Adobe ecosystem who needs to generate and align on large volumes of visual concepts before moving them into production. It’s the right call for designers, photographers, and creative directors doing serious concepting work that will ultimately land in Photoshop, Premiere, or Illustrator.

Choose Canva Magic Studio if you need fast, on-brand creative output and your users aren’t designers. Magic Studio’s Brand Kit integration means generated designs pull from your saved colors, fonts, and logos automatically: no cleanup, no manual adjustment. It’s not the tool for pixel-level creative control, but for marketing teams and content creators producing high volumes across multiple formats and channels, that’s rarely the priority.

Choose CoDesign if you need AI-generated output you can actually edit. Its output is editable design files, not flat exports: every element lands on the canvas as a structured design object (headline, image block, color field). That matters when the brief changes, the brand needs adjusting, or the output needs to become ten variations rather than one. For creators and teams producing branded content at scale, that editability is the difference between a starting point and a finished asset.

CoDesign is in early access. Be among the first to prompt a direction and walk away with a design you can actually use. Talk to our team to learn more.

What Is Vibe Design? The Definitive Guide for Product Builders, Designers, and Creative Teams

Klaudia — Fri, 20 Mar 2026 17:23:51 GMT

Vibe Design - A Concept That Just Got a Name

Creative work has always involved describing an idea and having someone skilled make it real. Vibe design follows the same principle, except the “someone” is AI, ready whenever you are, no brief needed. You describe what you want through text, images, brand assets, or sketches, and the AI generates the design. You direct the process; the tool handles the making.

The term itself is not new but was brought into mainstream use in early 2026 by Google via their Stitch announcement, and within a day it was appearing across The Register, CNBC, and TechRadar. The practice it describes had been building for well over a year, in Figma’s AI features, in generative design tools, in the growing number of products letting users create from a prompt rather than from a toolbox.

Today, we’ll define what vibe design actually is, trace where the idea came from, and explain what it means for people building products.

Where the Term Comes From: Vibe Coding’s Design Sibling

To understand vibe design, it helps to start with the concept it’s directly descended from: vibe coding.

Former Director of AI at Tesla and co-founder of OpenAI, Andrej Karpathy coined “vibe coding” in early 2025 to describe a shift in how developers work with AI-generated code. The idea: instead of writing implementation yourself, you describe what you want in natural language and an AI writes the code. The developer’s role shifts from implementation to direction. You stop thinking about how things are built and focus entirely on what you want to create.

Vibe design is the same shift applied to visual creation. Instead of opening a design tool and manually placing elements, adjusting colors, choosing fonts and spacing, you describe what you want (in words, or by uploading a reference image, a photo, a sketch, a brand kit), and the design takes shape. The phrase “vibe coding design” captures this lineage neatly: it’s the same intent-first principle, moved from engineering into the visual layer. The tool on the other side of that description is, increasingly, an AI design agent: software that takes the brief, does the multi-step work itself, and stays in the conversation for refinement.

The underlying movement had been gathering momentum since AI image generators went mainstream, accelerating rapidly as tools like Figma Make, Lovart, and now Google Stitch brought the concept directly into design workflows. The label arrived late to a trend that had already changed how a lot of creative work gets done.

What Vibe Design Actually Means: A Working Definition

Here is a working definition that holds up across tools and use cases:

Vibe design is a creative workflow in which the primary input is intent (described in natural language or visual references) rather than manual manipulation of design tools. The designer’s role becomes one of direction, curation, and refinement rather than construction.

Three things define a genuine vibe design workflow:

1. Intent-first input. The starting point is a brief, a description, a reference image, or a combination, not a blank canvas and a toolbox. You’re communicating what you want, not building it.

2. Generative execution. An AI interprets that intent and produces a designed output: a layout, a color scheme, a complete page, a set of variations. The construction step is handled by the system.

3. Human refinement in the loop. The human stays involved throughout, approving directions, adjusting outputs, steering away from things that don’t work. The AI handles execution; the human handles judgment.

What vibe design is not: it’s not simply using an AI image generator to produce pictures. Image generation is one possible input into a vibe design workflow, not the workflow itself. Vibe design produces editable, structured design outputs (layouts, components, documents, campaigns), not static images. The output is something you can work with, not just something you can look at.

How It Differs from AI-Assisted Design

“AI-assisted design” has covered a lot of ground over the past few years: autocomplete for design tokens, background removal, content generation within a layout. These are useful additions to a manual workflow. But in all of them, the designer still drives. AI is a tool called on for specific tasks while the human remains in the seat.

Vibe design flips the ratio. The AI drives the initial creation; the human steers and refines. It’s a different relationship with the tool, not a faster version of the same one. The clearest expression of that relationship is a design agent that returns editable files: you describe the direction, the agent builds the layout, and everything it produces stays open to manual editing afterward.

The distinction matters because it changes three things:

What skills are most useful. Writing clear, directed prompts and making fast curatorial judgments matters more than knowing every keyboard shortcut.
What the workflow looks like. You’re reviewing and steering outputs rather than constructing from scratch.
What software is relevant. Vibe design tools are built around a different interaction model than tools built to accelerate traditional manual design.

Vibe Design in Practice: Three Scenarios

The best way to make this concrete is to show what an AI design agent workflow built around vibe design actually looks like. These three scenarios cover the range of contexts where it’s becoming relevant.

Scenario 1: A Marketing Team, No Designer Available

A marketing manager at a mid-sized e-commerce brand needs a product launch campaign for social media. There’s no designer available this week. They’re tied up on a bigger project.

She opens the creative tool embedded in their marketing platform, uploads the product photo and the brand guidelines, and types: “Create a campaign for our summer collection, clean, minimal, white space heavy, headline-driven.”

She receives a set of formatted, brand-consistent assets sized for each social channel. The layouts are on-brand. The typography follows the guidelines she uploaded. She adjusts the headline copy on two of the assets and swaps one background color. The whole thing takes 12 minutes.

No design skills required. No third-party tool. No waiting for a designer to become available. The campaign goes out on schedule.

Scenario 2: A Designer Exploring Variations

A senior designer is working on a brand campaign for a luxury lifestyle client. She’s settled on a layout she likes, but she can’t land on the right color direction. Everything she’s tried manually feels either too cold or too safe, and exporting variations to compare them side by side is eating time she doesn’t have.

Instead, she types a single instruction into the agent chat: “Add a panel on the left with five color theme presets I can click to instantly apply to my design.”

The agent builds the panel directly inside the editor. Five named presets (Warm Sand, Midnight, Rose Quartz, Forest, Slate Blue), each applying a complete color theme across the entire design in one click: backgrounds, accents, headings, body text, all updated together. She works through all five in under a minute and finds the direction she was looking for without typing another prompt.

The variation-exploration workflow, at its most useful, doesn’t just produce more outputs; it builds the tools you need to make the decision faster.

Scenario 3: A Product Team Embedding Creative Capability

A print-on-demand platform serves retailers and small brands who need to produce product catalogues regularly but don’t have in-house design resource. One of their customers (a retailer for a furniture brand) opens the editor, pastes a CSV of five products into the agent chat, and describes the layout style she wants: two-column landscape, typography-led, minimal, referencing the aesthetic of Hay, Muuto, and Frama.

The agent generates a complete five-page catalogue inside the editor, one page per product, consistent layout throughout, with product names, descriptions, prices, and photo placeholders already in place. She follows up in plain language: “Pre-fill the photo placeholders with elegant product photography, make them black and white, soft contrast.” The agent updates all five pages. She adjusts one headline manually and exports.

The Vibe Design Tools Shaping the Space Right Now

Several tools are explicitly built around this workflow.

Tool	What it does	Where the agent lives	Best for
Google Stitch	Voice and text prompts to UI design	Google’s standalone tool	UI/UX designers, developers
Figma Make	Prompt to prototype inside Figma	Inside Figma (standalone)	Product designers working in Figma
Lovart	AI design agent for graphic creation	Lovart’s standalone platform	Marketing creatives, solo designers
IMG.LY	Design companion for all types of design works and tasks	IMG.LY Codesign studio	Designers, marketers, brand owners

Google Stitch is built around the idea that UI design should start with a conversation. You describe a screen (its purpose, the actions it needs to support, the general feel), and Stitch produces an interface design you can refine. It’s aimed at developers and UI/UX designers who want to move faster in the early stages of building a product interface. Where it works well is in getting from a rough idea to a structured screen layout without having to make every decision from scratch.

Figma Make extends the environment that product designers already work in. Because it lives inside Figma, it has access to your existing components, tokens, and design system. The prompt-to-prototype workflow is useful for designers who want to explore how a brief might translate into a working layout without manually composing every frame. Its biggest advantage is that the output lands directly in a space where a full design team can take over.

Lovart is focused on graphic and campaign creation rather than UI or product design. It’s built for the kind of work that marketing creatives and solo designers do a lot of, producing visual assets for social, campaigns, brand activations. The emphasis is on speed and aesthetic quality for graphic outputs rather than on structured, component-based design systems.

IMG.LY brings a design companion together with manual canvas edits through IMG.LY CoDesign - each element created by AI can be manually moved, changed, or adapted. It maintains brand and template context throughout the generation process, redefining what a template means: from a simple asset with placeholders to a true brand guideline. It also stands out for the breadth of its import and export options, which makes it a strong fit for graphic and asset design work.

Where Vibe Design Has Limits

Vibe design works well when there’s something to work from, like a reference image, a brand kit, or an existing visual direction. When there’s genuinely nothing to draw from, outputs tend toward the generic. A completely new brand identity with no existing visual language is a poor fit for a vibe design workflow; that kind of work still benefits from the deliberate, decision-by-decision process of traditional design.

It’s also less suited to accessibility-critical UI, where precise specification (contrast ratios, touch targets, interaction states) matters more than mood or aesthetic direction. A generated layout might look right without being accessible, and catching that requires careful manual review.

Finally, the more technically constrained the brief, the more refinement the output will need. Vibe design compresses the path to a starting point; it doesn’t always compress the path to a final, production-ready output. Teams that go in expecting to iterate will get more out of it than teams that expect to export and ship.

The Human Element: Vibe Design Is Not Autonomous Design

A common concern about vibe design workflows is that they reduce the role of skill and judgment in creative work. The evidence so far points the other way.

The most effective workflows keep the human firmly in control of direction, curation, and final judgment. The AI generates; the human decides what’s good, what fits the brief, what needs to change. The setup that makes this work in practice is one file, three authors: the agent, your code, and you all working in the same editable design instead of passing exports back and forth. Removing the human layer doesn’t improve outcomes; it just produces more output with no quality filter.

What changes is not whether human judgment matters, but at which stage it matters most. In a traditional design workflow, judgment is exercised continuously, at every click, every color choice, every alignment decision. In a vibe design workflow, judgment operates at a higher level: Is this the right direction? Does this match the intent? What needs to change?

The craft is still there. The instruments are different.

A Shift That’s Already Underway

Vibe design isn’t a trend arriving from the future. It’s a name for a shift that’s been building for several years and just became visible enough to label properly.

The creative AI tools exist. The workflows are being adopted. The user expectation is forming. Naming the practice in 2026 didn’t create the movement; it just gave it a shared vocabulary that makes it easier to talk about and build toward.

If what you’re looking for is a balance between manual control of the output and power of AI generation, IMG.LY Codesign might be the right fit for you. Talk to our team to see how it can fit inside your stack.

CE.SDK v1.69 Release Notes

Neslihan — Fri, 27 Feb 2026 18:29:53 GMT

v1.69 is the biggest developer experience release we’ve shipped. Agent Skills collapse the time from idea to working editor from days to under a minute. Production-ready Starter Kits eliminate days of boilerplate setup. PPTX and Canva Importer lets users bring their existing designs directly into your editor. And a comprehensive video overhaul across Web, iOS, and Android closes the gap between CE.SDK and native professional editing apps.

Let’s dive in!

Introducing IMG.LY Agent Skills for CE.SDK

The biggest change to CE.SDK’s developer experience since launch.

AI agents have changed how developers build. With Agent Skills, CE.SDK works natively with your AI agent.

Install the plugin for Claude Code or Cursor and your AI coding agent becomes a CE.SDK expert with bundled access to guides, API references, and starter kits across 10 web frameworks. No external services, no MCP servers, no context-switching.

What your developers can now do:

/cesdk:build — Describe a use case in plain language. The agent detects the framework, pulls the right starter kit, and scaffolds a working project autonomously.
/cesdk:explain — Ask any CE.SDK implementation question and get answers adapted to the specific framework and experience level.
/cesdk:docs-[framework] — Retrieve guides and API references directly from the IDE, without leaving the editor.

For your product, this means: developers prototype in minutes, not days. Non-technical team members can now build and extend functional editors independently. And onboarding new engineers to CE.SDK shrinks from a multi-day ramp to a single session.

→ Quick Start & Full Announcement
→ Explore Agent Skills Documentation

Launch in Minutes with Production-Ready Starter Kits for Web

Previously, getting CE.SDK running in a real product meant adapting complex demo projects: stripping features, adjusting architecture, and undoing assumptions baked into example code.

Starter Kits replace all of that with pre-configured, production-ready editor UIs.

Specialized kits for Photo, Video, and Design editors are scoped, clean, and immediately deployable. Each kit starts with a focused feature set. You explicitly opt in to what you need, so your UI stays performant and your users don’t see capabilities they can’t use.

For your product, this means: a functional, shippable editor in minutes rather than days of setup. Clean architecture from day one, without accumulating technical debt.

→ Explore Starter Kit Documentation

Let Users Bring Their PPTX & Canva Designs to CE.SDK

Some customers don’t start from scratch. They have existing creative assets locked inside PowerPoint and Canva, and until now, bringing those into CE.SDK required manual recreation.

The new PPTX Importer removes that barrier.

Users can import native PowerPoint files or export Canva designs as PPTX and bring them directly into your CE.SDK-powered editor, with layouts, elements, and structure intact.

→ Try the PPTX Import Demo
→ View PPTX npm

Professional-Grade Video Editing Across Platforms

This release expands CE.SDK’s video features across web & mobile platforms.

Control Video Speed (iOS & Android)

Users can now adjust the playback speed of video and audio clips via a dedicated Speed UI. This has been one of the most-requested mobile features and is now available on both platforms.
→ View Video Speed Documentation iOS
→ View Video Speed Documentation Android

Create Video Groups in the Timeline (Web)

Multiple clips can now be combined into a single manageable unit with synchronized timing, trimming, and a unified timeline representation.

This enables complex, multi-layer content production and speeds up the design process.

Video Timeline Now Grows Automatically (Web)

The timeline now automatically adjusts its height based on content. Manual track configuration is gone, and complex projects are easier to pick up for your new users.

Use a Clutter-Free iOS Timeline (iOS)

Non-video tracks now show a single thumbnail per clip, reducing clutter in multi-layer timelines on your iOS app.

Full Changelog

See all technical details, breaking changes, and performance improvements in the v1.69.0 Changelog.

Thank you for building with IMG.LY.

Introducing IMG.LY Agent Skills

Neslihan — Mon, 16 Feb 2026 18:54:06 GMT

TL;DR
We are moving beyond static documentation. IMG.LY Agent Skills is a specialized intelligence layer that transforms AI coding assistants (Claude Code, Cursor, Windsurf) into CE.SDK implementation experts. Instead of mapping APIs manually, you now inject our full SDK expertise directly into your IDE and coding agent.

→ Explore Agent Skill Documentation
→ Access Agent Skills GitHub Repository

The “Quick Start” Paradox

In our recent analysis of client integrations, a consistent pattern emerged: even with a high-performance SDK, the “Time to First Edit” is often delayed by research. Developers spend a significant portion of their initial integration time context-switching between their IDE and documentation, manually mapping complex API hierarchies to their specific framework.

We figured: You don’t just need better documentation; you need an expert partner who is already inside your codebase.

The Solution: IMG.LY Agent Skills for Web

We are moving beyond static documentation. Today, we are launching IMG.LY Agent Skills, a specialized intelligence layer that transforms AI coding assistants like Claude Code, Cursor, and Gemini into CE.SDK implementation experts.

Instead of searching for answers, you now give your agent our complete SDK knowledge with a single command.

How It Works: Your Autonomous Implementation Partner

By injecting versioned, live documentation and framework-specific starter kits directly into your agent’s context window, we’ve created three adaptive paths to launch:

1. The Explain Path

Architecture is complex. The Explain Skill acts as a digital implementation partner that adapts to you. It can provide a technical deep-dive on the export pipeline for a Senior Dev or a high-level summary for a Product Manager, in any language and at any level of detail.

2. The Build Path

Command your agent to “Add a photo editor to my React app.” The agent autonomously detects your environment, pulls the correct starter kit, and handles the boilerplate. It doesn’t just show you code; it builds the foundation for you.

3. The Docs Path

Stop the tab-switching. Your agent retrieves versioned API references offline, keeping your focus entirely within the terminal or IDE.

The Shift to Autonomous Engineering

This release marks a strategic shift for IMG.LY. We believe the next era of software development isn’t about “better documentation,” but about Portable Expertise.

By providing that AI with a “skill” rather than a manual, we are empowering teams to move from a multi-month build cycle to a 30-second scaffold.

Supported Frameworks

The initial Web release supports 10 frameworks out of the box:
React · Vue.js · Svelte · Angular · Next.js · Nuxt.js · SvelteKit · Electron · Node.js · Vanilla JS

Get Started

Run the command:

npx skills add imgly/agent-skills

For example, Claude Code users, can run:

claude plugin marketplace add imgly/agent-skills

claude plugin install cesdk@imgly

→ Explore the full Agent Skill Documentation

The “editor of your dreams” is now a conversation away.

Join 3,000+ creative professionals who get early access to new features and updates. Subscribe.

Build in a Day: AI Video Clipping with CE.SDK

Eray — Thu, 05 Feb 2026 12:17:28 GMT

Introduction

We built a video shortener in a single day using Claude Code and CE.SDK. It extracts 3-4 short clips from long-form video, handles transcription, identifies the best moments via AI, detects speakers, and outputs vertical/horizontal/square formats, all running in the browser.

Features:

Extracts 3-4 clips per video (highlights, summaries, or cleaned-up edits)
Outputs 9:16 (vertical), 16:9 (landscape), or 1:1 (square)
Detects speakers and maps them to faces with user confirmation
Auto-crops to follow the active speaker
Adds captions and text hooks
Non-destructive: change aspect ratio or template without re-processing

Best suited for: Videos with speech/dialogue (podcasts, interviews, presentations, vlogs)

Why Client-Side?

CE.SDK’s CreativeEngine runs in the browser via WebAssembly. Video decoding, timeline manipulation, effects, and preview all happen on the user’s device.

Benefits:

No upload/download wait: edits preview instantly
Non-destructive: switch aspect ratio or template without rendering
Lower infrastructure costs: your costs don’t scale with video length or user count

Tech Stack

Frontend: Next.js + React
Video Engine: CE.SDK (CreativeEngine)
Transcription: ElevenLabs Scribe v2
AI Analysis: Google Gemini

Architecture Overview

High-Level Flow

Required API Keys

Service	Purpose	Environment Variable
CE.SDK	Video editing engine	`NEXT_PUBLIC_CESDK_LICENSE`
ElevenLabs	Speech-to-text transcription	`ELEVENLABS_API_KEY`
Gemini (via OpenRouter or direct)	AI highlight detection	`OPENROUTER_API_KEY` or `GEMINI_API_KEY`

Setting Up CE.SDK

What is CE.SDK?

CE.SDK (CreativeEngine SDK) is a browser-based engine for video, image, and design editing: a programmable video editor you can embed in your app.

Key Concepts:

Engine: The runtime that manages the editing session
Scene: The document/project containing all elements
Blocks: Individual elements (video clips, text, shapes, audio)
Timeline: Time-based arrangement of blocks for video editing

Installation

npm install @cesdk/cesdk-js

Initializing the CreativeEngine

import CreativeEngine from '@cesdk/cesdk-js';

const engine = await CreativeEngine.init({
  license: process.env.NEXT_PUBLIC_CESDK_LICENSE,
});

// Create a video scene
const scene = engine.scene.createVideo();

// Get the page (timeline container)
const pages = engine.scene.getPages();
const page = pages[0];

// Configure page dimensions for your target aspect ratio
engine.block.setWidth(page, 1080); // 9:16 vertical
engine.block.setHeight(page, 1920);

Uploading Video to CE.SDK

CE.SDK works with video through a fill-based system. The graphic block is the container, while the video fill holds the actual media source and playback properties.

// Create a video block
const videoBlock = engine.block.create('graphic');
const videoFill = engine.block.createFill('video');

// Set the video source
engine.block.setString(
  videoFill,
  'fill/video/fileURI',
  videoUrl // Can be a blob URL or remote URL
);

// Apply fill to block
engine.block.setFill(videoBlock, videoFill);

// Add to timeline
engine.block.appendChild(page, videoBlock);

Extracting Audio for Transcription

// Configure audio-only export
const mimeType = 'audio/mp4';

// Export just the audio track
const audioBlob = await engine.block.export(page, mimeType, {
  targetWidth: 0,
  targetHeight: 0,
});

// audioBlob can now be sent to transcription API

Setting both dimensions to 0 tells CE.SDK to skip video encoding entirely, making this export much faster than exporting the full video.

Getting Video Metadata

// Get video duration
const duration = engine.block.getDuration(videoBlock);

// Get dimensions from the fill
const videoFill = engine.block.getFill(videoBlock);
const sourceWidth = engine.block.getSourceWidth(videoFill);
const sourceHeight = engine.block.getSourceHeight(videoFill);

console.log(`Video: ${sourceWidth}x${sourceHeight}, ${duration}s`);

AI-Powered Transcription & Highlight Detection

The Pipeline

Audio → Transcription: Send extracted audio to ElevenLabs Scribe
Transcription → Analysis: Feed word-level transcript to Gemini
Analysis → Timestamps: Map AI suggestions back to precise video times

Transcription with Speaker Diarization

ElevenLabs Scribe v2 provides:

Word-level timestamps (start/end time for each word)
Speaker diarization (which speaker said what)

The output is a structured transcript where each word has a precise timestamp, enabling frame-accurate editing.

AI Highlight Detection with Gemini

The prompt structure matters. Here’s what works:

You are analyzing a video transcript to identify segments for short-form content.

TRANSCRIPT:
[Word-by-word transcript with timestamps]

TASK:
Identify 3-4 segments that work as standalone short videos. For each segment:
1. Find the exact starting and ending words
2. Ensure clean sentence boundaries (no mid-sentence cuts)
3. Aim for 30-60 second segments

OUTPUT FORMAT (JSON):
{
  "concepts": [
    {
      "id": "concept_1",
      "title": "Hook title",
      "description": "Why this segment works as a standalone clip",
      "trimmed_text": "The exact transcript text to keep...",
      "estimated_duration_seconds": 45
    }
  ]
}

CRITERIA FOR SELECTION:
- Strong hooks (surprising statements, questions, bold claims)
- Complete thoughts (don't cut mid-explanation)
- Emotional peaks (humor, insight, controversy)
- Standalone value (makes sense without context)


Before finalizing each segment, ask: "If someone started watching here,
would they understand what's being discussed?"

Mapping Back to Timestamps

Once Gemini returns the trimmed_text, we match it against our word-level transcript to find exact timestamps:

AI returns:     "The secret to success is actually quite simple..."
Transcript has: [{ word: "The", start: 45.2 }, { word: "secret", start: 45.4 }, ...]

Result:         Trim video from 45.2s to 52.8s

This text-matching approach is more reliable than asking the AI to output timestamps directly. LLMs can hallucinate timestamps or miscalculate offsets, but they’re excellent at identifying the right words, so we let the transcript data provide the ground truth for timing.

Working with the CE.SDK Timeline

Understanding Blocks

// Video/Image content
const graphic = engine.block.create('graphic');

// Audio track
const audio = engine.block.create('audio');

// Text overlay
const text = engine.block.create('text');

// Each block can be positioned on the timeline
engine.block.setTimeOffset(block, startTimeInSeconds);
engine.block.setDuration(block, durationInSeconds);

Manipulating Trim Points

Trimming controls which portion of the source media is shown:

const videoFill = engine.block.getFill(videoBlock);

// Set where in the source video to start (in seconds)
engine.block.setTrimOffset(videoFill, 45.2);

// Set how long to play from that point
engine.block.setTrimLength(videoFill, 30.0);

// Also update the block's duration to match
engine.block.setDuration(videoBlock, 30.0);

Working with Fills and Their Timing

// Get the fill (contains the actual media)
const fill = engine.block.getFill(block);

// Fills have their own timing properties
const trimStart = engine.block.getTrimOffset(fill);
const trimDuration = engine.block.getTrimLength(fill);

// The block's duration should typically match the fill's trim length
engine.block.setDuration(block, trimDuration);

Think of the fill as the media source (which part of the original video to use) and the block as the timeline placement (when and how long it appears). Both need to be updated together for clean edits.

Creating Time-Based Edits from Transcript Words

interface TranscriptWord {
  word: string;
  start: number;
  end: number;
  speaker_id?: string;
}

function applyTranscriptTrim(
  engine: CreativeEngine,
  videoBlock: number,
  words: TranscriptWord[]
) {
  if (words.length === 0) return;

  const startTime = words[0].start;
  const endTime = words[words.length - 1].end;
  const duration = endTime - startTime;

  const fill = engine.block.getFill(videoBlock);

  engine.block.setTrimOffset(fill, startTime);
  engine.block.setTrimLength(fill, duration);
  engine.block.setDuration(videoBlock, duration);
}

Generating Speaker Thumbnails

async function generateSpeakerThumbnail(
  engine: CreativeEngine,
  videoBlock: number,
  timestampSeconds: number,
  size: number = 128
): Promise<string> {
  const fill = engine.block.getFill(videoBlock);

  // Seek to the specific timestamp
  engine.block.setTrimOffset(fill, timestampSeconds);
  engine.block.setTrimLength(fill, 0.1); // Just a single frame

  // Export as image
  const blob = await engine.block.export(videoBlock, 'image/jpeg', {
    targetWidth: size,
    targetHeight: size,
  });

  return URL.createObjectURL(blob);
}

We sample multiple timestamps throughout each speaker’s talk time to show different facial angles and expressions. This helps users identify the right person even if they’re looking away or mid-gesture in one frame.

Speaker Detection & Face Tracking

Why Semi-Automatic?

Fully automatic speaker detection fails often enough that we added a confirmation step. Users verify detected faces against speaker names from the transcript. It takes a few seconds and prevents bad crops on the entire video.

How It Works

Sample frames throughout the video
Detect & cluster faces using face-api.js (runs in browser, no server needed)
User confirms speaker identities via thumbnails
Correlate with transcript diarization to map speakers → face locations

This gives us verified speaker-to-face mapping for dynamic cropping and picture-in-picture layouts.

Multi-Speaker Templates & Dynamic Switching

The Concept

When a video has multiple speakers, we can create layouts that show:

The active speaker prominently
Other speakers in smaller picture-in-picture views
Dynamic switching as the conversation flows

Creating Picture-in-Picture with CE.SDK

// Duplicate the video block for each speaker slot
const pipBlock = engine.block.duplicate(originalVideoBlock);

// Position and size the PiP
engine.block.setWidth(pipBlock, 200);
engine.block.setHeight(pipBlock, 200);
engine.block.setPositionX(pipBlock, 20); // 20px from left
engine.block.setPositionY(pipBlock, 20); // 20px from top

// Enable cropping
engine.block.setClipped(pipBlock, true);
engine.block.setContentFillMode(pipBlock, 'Cover');

Key Technique: Muting Duplicate Audio

When duplicating video blocks for multi-speaker layouts, each copy has its own audio track. We must mute all but one. The setMuted API operates on the video fill, not the block itself:

// For each speaker slot after the first, mute the video fill
if (slotIndex > 0) {
  const videoFill = engine.block.getFill(duplicatedBlock);
  if (videoFill) {
    engine.block.setMuted(videoFill, true);
  }
}

Dynamic Speaker Switching

As the active speaker changes throughout the video, we:

Detect which speaker is talking (from transcript diarization)
Swap speaker positions in the template
Keep the active speaker in the prominent position

The layout updates automatically as the conversation switches between speakers. We apply different trim offsets to each duplicated block based on the transcript timing, so the main speaker slot shows the person currently talking while PiP slots show the listeners.

Preview, Playback & Export

Setting Up the Canvas

const container = document.getElementById('cesdk-canvas');
engine.element.attachTo(container);

Playback Controls

engine.player.play();
engine.player.pause();
engine.player.setPlaybackTime(30.5); // seek to 30.5 seconds

const currentTime = engine.player.getPlaybackTime();
const isPlaying = engine.player.isPlaying();

Syncing UI State

engine.player.onPlaybackTimeChanged(() => {
  const time = engine.player.getPlaybackTime();
  updateTimeDisplay(time);
  updateProgressBar(time / totalDuration);
});

engine.player.onPlaybackStateChanged(() => {
  updatePlayButton(engine.player.isPlaying());
});

Export Options

const exportOptions = {
  targetWidth: 1080,
  targetHeight: 1920,
  framerate: 30,
  videoBitrate: 8_000_000, // 8 Mbps
};

const blob = await engine.block.export(
  page,
  'video/mp4',
  exportOptions,
  (progress) => updateProgressBar(progress * 100)
);

// Trigger download
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = 'shortened-video.mp4';
a.click();

For longer videos, consider showing estimated time remaining or allowing background export. Browser export is single-threaded and blocks the tab. A 5-minute export of a 60-second clip isn’t unusual on average hardware, so user feedback is critical.

The Finished App

The user flow:

Upload → Drop a long-form video into the browser
Configure → Pick output mode (highlights/summary/cleanup) and aspect ratio (9:16, 16:9, 1:1)
Verify speakers → Match detected faces to transcript speaker names
Review clips → Browse the 3-4 AI-suggested segments, adjust if needed
Choose template → Solo speaker, sidecar, stacked, etc.
Preview → Scrub through the timeline, see exactly what you’ll get
Export → Download the final video directly from the browser

What’s Next

Ideas for Extension

Caption style controls: Custom fonts, animations, and positioning for subtitles
B-roll insertion: Automatically add relevant stock footage
Music & sound effects: AI-selected background audio
Brand templates: Custom overlays, intros, outros
Batch processing: Process multiple videos in sequence

Taking It Server-Side

Client-side processing for large files strains browser memory, and users must keep the tab open during export. A hybrid approach works better for production: upload in the background while users edit, then render on a server. You can also offload just the export step: let users build their edits in the browser, then send the CE.SDK scene JSON to your backend for faster, background rendering.

CE.SDK runs server-side with the same API. For batch processing, background jobs, or offloading rendering from user devices, see the CE.SDK Renderer for creative automation.

Resources

Made by IMG.LY with CE.SDK

From Prompt to Editor: Running CE.SDK Inside ChatGPT with the Apps SDK

Jan — Fri, 19 Dec 2025 15:43:16 GMT

With the new ChatGPT Apps SDK and Model Context Protocol (MCP), chat interfaces are starting to look less like Q&A tools and more like places where work actually happens. To explore what that means for creative workflows, we built a small technical demo: CE.SDK running directly inside ChatGPT.

From a user’s perspective, the flow is almost trivial. You ask ChatGPT for something like an ecommerce template. ChatGPT searches our template catalog, selects a matching design, and opens it instantly in a fully interactive CE.SDK editor, right inside the chat interface. What looks like a preview is, in fact, a real editor loaded with a real template scene.

This isn’t meant as a product announcement. It’s a technical proof of concept showing how creative SDKs can plug directly into AI-native interfaces.

CE.SDK as a ChatGPT App

The integration is built around a custom MCP server that exposes CE.SDK to ChatGPT as a tool. The server speaks OpenAI’s JSON-RPC–style MCP and implements the standard lifecycle methods (initialize, tools.list, tools.call, resources.read). It knows about our premium template catalog and emits structured payloads that the frontend understands.

On the client side, a Next.js app listens to tool output events streamed from ChatGPT, renders CE.SDK widgets, and hydrates them with the payloads returned by the tool, such as a scene URL, placeholder values, or export permissions. Templates are loaded via CE.SDK’s Template API, either from a URL or from a serialized scene string.

Under the hood, the stack is fairly conventional:

Next.js 15 (App Router)
CE.SDK Web / CreativeEngine
A custom MCP handler to normalize JSON-RPC
Vercel for hosting

What’s new is not the technology itself, but the context in which it runs.

Working with MCP in Practice

The hardest part of the demo wasn’t CE.SDK. It was MCP.

OpenAI’s MCP implementation is extremely strict. Even the smallest schema mismatch can trigger the infamous “TaskGroup 424” error, usually without any hint as to what went wrong. In many cases, the HTTP response is technically successful, but the JSON structure doesn’t match the expected schema closely enough.

The key lesson here is to treat MCP responses as hard contracts:

Validate every response against a schema (for example with zod).
Mirror OpenAI’s field names exactly, even for empty or optional capabilities.
Assume that a 424 almost always means “your JSON shape is wrong”.

Another important insight was how critical visual context is in chat-based tools. If your MCP responses don’t include thumbnails or preview images, ChatGPT will often fall back to rendering links. For creative tools, that immediately breaks the experience. In a chat UI, visuals aren’t an enhancement. They are the interface.

State handling also requires a shift in mindset. ChatGPT can replay tool calls, and each prompt effectively creates a new widget instance. You can’t rely on mutating an existing editor. The frontend needs to be idempotent: load scenes from serialized state first, then apply changes. Every tool call should be treated as a fresh render.

Why This Pattern Matters

This demo points to a broader change in how creative software may be accessed. Chat becomes a coordination layer, not just a conversational one. Instead of explaining how something could be designed, the AI opens the actual editor and lets the user continue from there.

For CE.SDK, this fits naturally. Editors become embeddable capabilities rather than standalone applications, and AI systems become the entry point into creative workflows. Prompting turns into doing.

Beyond OpenAI: The MCP UI Standard

Although this demo uses OpenAI’s MCP, the architecture maps cleanly to the new MCP UI standard recently introduced by Anthropic. That standard aims to make tool definitions and UI rendering more consistent across models and platforms.

Because this integration already separates tool logic from UI rendering and relies on structured, explicit payloads, transferring it to Anthropic’s MCP UI model is conceptually straightforward. CE.SDK can act as a reusable creative surface across ChatGPT, Claude, and future AI app ecosystems.

This demo is intentionally small and technical, but it highlights a meaningful shift: AI systems that don’t just describe creative outcomes, but open the tools to actually create them.

Animate Between Images - AI-Native Video Workflows with CE.SDK and Veo 3

Jan — Wed, 22 Oct 2025 14:10:31 GMT

With the release of Veo 3.1, we wanted to show just how effortless it is to embed generative AI capabilities directly into creative workflows. In this quick demo, we integrated Veo 3 into CreativeEditor SDK (CE.SDK) enabling users to animate between two still images in just a few clicks.

From Still Images to Motion

In our demo, we start with two images of the same person, one wearing a hat, the other without. Inside the editor, users simply select both images, click the AI context button, and choose “Animate between images.”

The images are loaded into a side panel where users can optionally add a text prompt to guide the transition. Once generated, the resulting short video is placed directly on the canvas ready for editing, compositing, or export.

What’s particularly impressive is the generation speed. In this example, Veo 3.1 produced a smooth 8-second transition in just 9 seconds a major improvement compared to earlier versions. This speed makes iterative creative workflows feel fluid and responsive, bridging the gap between prompt-driven generation and real-time editing.

Try It Out

You can check out the implementation on GitHub and give Veo 3.1 a spin inside your CE.SDK instance (sign up for a free trial if you haven’t already).

A Glimpse of AI-Native Editing

This simple feature highlights how easy it is to make your editor AI-native, combining traditional editing tools with generative intelligence.

Practical use cases include:

E-commerce
Show products “in action” or animate between styles and configurations.
Marketing
Create quick product reveal animations from static assets.
Content creation
Generate short motion clips or “tween” between creative scenes.

And if users want longer clips, they can simply add another 8-second track and transition seamlessly into the next.

The Future of Creative Workflows

With Veo 3 integrated, CE.SDK becomes a powerful playground for AI-driven creativity from image-to-video to scene interpolation and contextual animation.

We already empowered over 600 innovative startups, government entities, and Fortune 500 companies to add powerful design, video, and photo editing workflows to their products. Get in touch, to see how we can do the same for you.

What is Visual Prompting?

Jan — Tue, 29 Jul 2025 13:06:11 GMT

A New Paradigm for Creative AI, Built by IMG.LY

To say it’s trite to refer to the impact of AI in this or that domain as disruptive or groundbreaking would be an understatement. Yet, few areas have been as profoundly affected as the creative process. With just a text prompt, anyone can produce stunning images, remix visual styles, and explore design possibilities at a scale and speed never seen before. AI has inserted itself so quickly into this process that its gone from curious novelty to an essential part of the creator toolchain.

The more serious adoption we see, however, the more key limitations of today’s AI tooling come into focus: the prompt itself.

Text alone, for all its expressive power, struggles to capture the essence of visual intent. Most creative work doesn’t begin with a sentence it begins with a sketch, a layout, a mood board, or an arrangement of elements. Visual ideas are shared by pointing, placing, showing.

At IMG.LY, we have begun to think about better ways to direct AI for visual generation, the term we use is Visual Prompting.

Visual Prompting: the practice of composing a visual scene or layout as input for a generative model.

Instead of describing what you want with paragraphs of text, you show it directly using a canvas of images, text, spatial cues, and annotations. This visual composition then becomes the prompt for the AI to generate new content in return. It’s a more natural, intuitive, and powerful way to collaborate with AI, especially when integrated directly into the creative process.

Problem: the Chat Disconnect

The current generation of AI tools has largely been shaped by language-first interfaces. Whether it’s ChatGPT for writing or Midjourney for image generation, the assumption is the same: the user will type a descriptive prompt, and the AI will generate a result based on it.

But when it comes to design, this workflow quickly runs into friction. Visual ideas are inherently spatial and non-linear. Trying to express layout, balance, mood, or specific spatial relationships through text can feel like trying to describe a painting over the phone. It’s possible but unnecessarily cumbersome.

A designer might want to:

Indicate that a certain area in the image should be blue.
Replace a background with a texture sample.
Position a character precisely in a composition.
Annotate which parts of a scene to preserve or modify.

All of these are difficult to express fluently in text. But they’re effortless in a visual interface. The truth is: an image is worth more than a thousand words when prompting an image.

What Is Visual Prompting?

Visual Prompting is a multimodal approach to generative AI, where the input to the model is not just text, but a full visual composition: images, text, annotations, and layout.

Rather than prompting AI in isolation, the user builds their intent on a canvas. This might include:

Reference images that communicate mood or style.
Text blocks indicating desired copy or instructions.
Annotations pointing to specific areas with notes like “make this glow” or “replace this object.”
Spatial composition: where elements are arranged meaningfully to convey intent.

The visual prompt is then interpreted by a multimodal model such as OpenAI’s gpt-image-1 to generate new visual content that reflects not only the textual description, but also the visual context.

How Visual Prompting Works in CE.SDK

About time for an example. As part of our recent AI released we demoed how to use OpenAIs gpt-image-1 model to build visual prompting into CreativeEditor SDK (CE.SDK).

Here’s what the process looks like inside CE.SDK:

Compose Visually: The user creates a layout with reference content, uploaded images, icons, color schemes, design elements, placeholder text, and annotations. This composition represents the “prompt” in visual form.
Add AI Layers: With a single click, the user can trigger image generation using CE.SDK’s built-in AI plugin. The plugin sends the visual context (alongside any optional text input) to a multimodal model capable of interpreting both.
Refine and Iterate: Users can adjust the layout, reposition elements, change annotations, or layer in new references, then prompt again. Because the canvas is interactive and editable, the feedback loop is tight.
Build Up Complexity: Over time, users can layer generated images with manually designed components or other generated outputs, creating rich compositions that blend AI creativity with human direction.

This workflow turns the traditional prompt/response cycle into a conversation between the designer and the model, with the canvas acting as the shared language.

Who Is Visual Prompting For?

The use cases for Visual Prompting extend across industries:

Creative teams can go from reference to generation in seconds, iterating visually instead of wrangling prompts.
Marketing teams can generate regionalized or personalized creative variants from a shared layout.
Product designers can prototype in context, turning layouts into realistic screens without leaving the editor.
Storytellers and content creators can use annotated sketches to generate detailed illustrations or scene variations.
E-commerce platforms can give sellers the power to visually customize their brand materials with AI assistance.

In every case, Visual Prompting replaces friction with flow and text-based prompting with something more expressive, more reliable, and more fun.

Built for This: Multimodal Models and CE.SDK’s Plugin System

Visual Prompting is only possible because of two parallel advancements:

Multimodal AI models, such as OpenAI’s gpt-image-1, that can interpret both images and text, understand spatial relationships, and respond to annotated cues.
A flexible, composable editor SDK like CE.SDK, which enables the construction of visual prompts on a live canvas, and makes it easy to integrate AI models directly into the design flow.

Our SDK was built from the ground up to support AI-first creative workflows. Its plugin architecture allows you to add any model or API, image generation, video generation, captioning, text rewriting and use it natively inside the editor without the need to switch tools or copy/paste.

Generative AI’s full potential is only unlocked when it is embedded directly into the tools creatives use not siloed in chatbots or separate interfaces. Visual Prompting allows that embedding to go even deeper, aligning the mode of input (visual) with the desired output (visual).

Explore It Yourself

🎨 Try out Visual Prompting in our AI Editor demo
📘 Learn How to Integrate AI into CE.SDK
💬 Contact Us to Bring Visual Prompting to Your Product

Vibe-Engineering: When AI Does All the Coding, What Do We Actually Do?

Daniel — Mon, 07 Jul 2025 09:51:05 GMT

This isn’t an article about AI replacing engineers, it’s about discovering what (software) engineering will become when freed from the mechanics of coding itself.

Vibe-Coding, and Vibe-Engineering are omnipresent in my social feeds these days. As with every new trend, it’s hard to judge what really works and what is pure marketing hype. But the central question nagged at me: if AI really can do all the coding, what exactly are we humans supposed to do?

Therefore, I wanted to find out for myself. I didn’t want to build just a Hello World example, so I pulled an old idea out of the closet and dusted it off.

The Experiment: AI Agents Code, Humans Curate

For some time, I had considered porting our IMG.LY background removal library from JavaScript to other platforms using Rust. However, I had postponed this side project due to the anticipated effort involved. This seemed like the perfect project to discover what my role would be in a world where AI does the heavy lifting. To pull it off, I self-imposed only one strict rule:

Hands off the code!

Every commit, every fix, every feature would be handled by an AI coding agent. My role? Being the curator, formulating my intent, reviewing, play-testing, and guiding the agent with feedback.

The agent should be the sole coder, from start to finish.

This was a true hands-off experiment: could an AI build an entire Rust library if I never touched the code?

Is it really something new? After years of handing off development to teammates as a CTO or teaching students, this didn’t feel completely different.

So, what happens when you let an AI do all the coding, and you just review and give feedback? What does it actually mean to be a human in this new paradigm?

The result is baffling and gives a glimpse of what we as engineers can expect for the future.

10,000+ lines of production-ready code. Authored by AI, orchestrated by a human.

Seeing is believing? Then explore the results and judge for yourself at github.com/imgly/background-removal-rs.
I open-sourced the full code and the agent rules used during this experience.

Here’s what I learned: vibe-engineering an entire Rust library with a coding agent CLI as my only pair programmer.

On a quick note, I chose to use Claude Code with Opus 4 and Sonnet 4, as I was already familiar with them and had achieved the best results so far. The costs were kept in check by using a Claude Max Account, which at the time is around ~€100.

The General Workflow That Emerged

After some initial time to get used to working with an AI agent, I realized that the workflow follows pretty closely a typical software engineering flow, but with an important twist that answers our central question: what do humans actually do when AI codes? While most people focus on the code writing part, I discovered that the quality assurance part becomes the human’s primary domain. In my experiments, I realized that to keep the code intact over time, this QA phase is essential, and this is where humans truly shine. Most interestingly, with the right rules and prompting, we can let the agent do the heavy lifting here, too.

Here’s how the collaboration actually worked in practice:

Getting Started
The project began with me setting up the environment and configuring timeouts for long-running tasks, because AI agents need time to compile Rust code and run extensive test suites. Together, the AI and I created a CLAUDE.md file with coding standards and rules (you can see the actual rules we used). We also wrote project requirements documents collaboratively, with me providing the vision and the AI helping structure the technical details.

The Development Dance
Once we got rolling, a natural rhythm emerged. I’d describe what I wanted, something like “add support for different image formats with color profile preservation.” The AI would then create a detailed implementation plan, breaking down the work into manageable chunks. I’d review this plan, often tweaking the approach or scope, then give the green light.

What happened next was fascinating: the AI would write the code, run correctness checks, update all the tests (unit tests, documentation tests, end-to-end tests), run the linter, analyze coverage, and even update documentation and examples. Sometimes it would run performance benchmarks without me asking, the overzealous builder at work.

My New Role: The Quality Guardian
My job became verifying that the intent was met and the API actually made sense. I’d play-test the functionality, try to break it in ways a real user might, and provide feedback. When something worked well, I’d document what we learned for future development. The AI would then write tests to lock in that new functionality, ensuring we never accidentally broke what was working.

This cycle repeated for every feature, with the AI handling the heavy lifting while I focused on direction, quality, and user experience.

The interesting part was that the more I got used to the workflow, the more I handed off to the agent. The validation and general experience had to be largely influenced or orchestrated by me, but building, testing, and even updating documentation could be delegated. In the beginning, I started just building things, but the more we built, the more the AI agent failed to keep existing functionality intact. With those small context windows, it’s kind of understandable after all.

Quality Assurance & Knowledge Capturing Matters More Than Ever

As engineers, we know that at some points we cannot grasp every influence of our code changes on the whole code base anymore. Therefore, we use tests and tooling to help us keep our sanity.

The good thing is that the Rust and Cargo toolchains are exceptionally high quality when it comes to providing the right tooling also for agents.

Due to its lack of long-term context and limited knowledge of all the library’s capabilities, the tests and verification mechanisms become crucial. While these are part of the context, they’re not stored in the agent’s memory but baked into the codebase itself. With the tests and the tooling being available to the agent, it will still make mistakes, as I do, but they can effectively auto-heal or auto-correct their own mistakes with the provided feedback.

I can only advise being even stricter with QA from the very beginning.

Another thing that became apparent very quickly is that, unlike a human colleague with lots of context and a good memory, agents still need significant help to remember things to avoid rediscovering the same information repeatedly. This revealed another crucial human role: context engineering becomes critically important. As such, we engineers become the keepers of institutional knowledge, the ones that help the agent remember what worked and what didn’t. I would describe it as knowledge capture, for lack of a better term.

Tips & Tricks

AI Is Eager, Resourceful, and Has Memory Like a Sieve

Here are the key insights I gained from building the library with an AI coding agent:

Code that works: This might seem obvious, but it wasn’t clear to me if the library would ever be usable or publishable, but it is. The AI proved it could deliver production-ready code.

Time Effectiveness: While creating the library took around 3 weeks, don’t let this fool you. Most of the time, the agent worked alone, and I only had to check in once in a while to review and give feedback. Iteration cycles and rewriting still take time. So I wouldn’t say development was faster than if I did it myself, but much more scalable and effective with my own time. Here’s what I actually spent my time on: reviewing architectural decisions, ensuring the API made sense, and play-testing the final product.

Stamina: The agent never complained, even when fixing 400+ warnings after setting the linter to pedantic. It pushed through. The AI powered through hundreds of warnings, even clustering them and suggesting which to fix first. To be fair, most humans would have either gone crazy or aborted the effort claiming “it’s good enough.” This revealed something important: as humans, we don’t need to be the ones grinding through tedious tasks anymore.

Overzealous Builder: The agent often built more than I asked for. Scoping and clear implementation plans were essential. This taught me that one of the key human roles is setting boundaries and maintaining focus.

Premature “Done”: Sometimes the agent claimed it was finished but left code mocked or incomplete. I learned to always ask, “Is there anything left to implement?” Quality assurance becomes fundamentally a human responsibility.

Old Knowledge: The AI sometimes relied on outdated library knowledge. I had to remind it to check the latest docs and version of dependencies, but when asked, it searched the web, GitHub, crates.io, etc., and gathered all necessary information for using a library. Humans become the validators of information freshness.

Forgetting and Compaction: Larger features grow outside the context window, so the agent would “compact” context, sometimes forgetting important details. Forcing it to write implementation plans in markdown files helps maintain flow and allows better resuming and bookkeeping. We humans become the keepers of long-term project memory.

Helps Establishing Rules: While working with the agent, I saw repeating patterns like it forgetting how to check, build, and test the application, or best practices like using git worktrees. I started adding new rules to Claude.md by myself at first, but quickly realized that the agent is far better at formulating and adding new rules if I asked it to. The human role evolved into being the pattern recognizer and rule creator.

Doesn’t Always Obey the Rules: Most of the time, the rules are followed, but occasionally it doesn’t follow them and forgets to automatically run all tests. This reinforced that humans must remain the final guardians of process and quality.

Code Deletion: Occasionally, the AI “fixed” issues by removing important code paths. Solution: Insist on never removing functionality without discussion, and build a robust test suite. We become the protectors of existing functionality.

Process Management: The AI struggled with background processes, understanding when to run things in the background, and keeping track of what it started. For example, starting a web server via Bash (see Claude Code Bash tool docs), and then trying to use a playwright tool (see also playwright-mcp) to access this server was blocked by these issues. Complex orchestration remains a distinctly human skill.

Overly Agreeable: Too often, it seems to just agree with whatever I said. For the future, I wish agents wouldn’t be so obedient. This highlighted that humans need to be the challengers and devil’s advocates in the process.

Further Remarks

After three weeks of this experiment, the patterns became clear. Here’s what I learned about the human role in AI-assisted development:

Ensure Verify, and Self-Correct: Always ask the AI to check, format, lint, test, and benchmark its work. Your job becomes quality orchestration, not quality execution.

Always Plan First: Insist on a clear implementation plan before coding begins. Humans excel at high-level architectural thinking and breaking down complex problems.

Write It Down: Let an agent keep notes, todos, and open issues in markdown files. Ideally in the repository itself. Don’t rely on the AI’s memory. We become the institutional memory keepers.

Scope Features: Break tasks into small, manageable pieces. Feature scoping and boundary setting become a core human skill.

Test Suite is King: A robust test suite prevents accidental “fixes” that remove functionality. Humans become the guardians of existing behavior.

Stay Involved: Fast iteration and discussion with the AI is crucial. Don’t go fully hands-off. Active curation and guidance remain essential.

Conclusion

So, what do humans actually do when AI does all the coding?

To answer that, let’s look at what AI agents are today:

Agents are super-talented, overly obedient coding assistants with lots of stamina and endless potential.

They’re fast, never complain, and iterate like champs. But they need your guidance, structure, and a healthy dose of skepticism.

In the end, the best results come from a true partnership: you plan and guide—the AI builds, fixes, and learns.

If you follow these concepts, then

“Vibe engineering is a highly engaging experience.”

And if you’re wondering: yes, I’d do it again. But next time, I’ll make sure the AI writes everything down too, and I’ll put even more effort into validating and securing new features as soon as they’re play-tested and verified. The human role isn’t disappearing. It’s evolving into something more strategic and impactful.

Through this experiment, I’ve identified three fundamental shifts that define what engineering becomes in an AI-assisted world:

From Execution to Orchestration
We’re no longer the ones typing code. We’re the conductors. Our new skills revolve around prompting (framing tasks so AI understands our intent), directing workflows (knowing when to use AI versus human judgment), and tool fluency (combining different AI tools to create something greater than their parts).

From Knowledge to Reasoning
AI can recall facts and syntax better than any human ever could. But what matters now is our ability to interpret, contextualize, and make sense of uncertainty, something current AI still struggles with. We become the ones who ask “why” and “what if” rather than just “how.”

From Repetition to Adaptation
Skills rooted in routine are increasingly automated. The enduring value lies in adaptive thinking, problem-framing, and reinventing approaches when the usual patterns don’t apply. We’re the ones who recognize when something feels off, even if we can’t immediately articulate why.

Afterthought

During this experiment, I saw that the agent repeated similar tasks over and over again. One thing that struck me was looking up documentation. Whenever something with a third-party library didn’t pan out, it started web-searching for info and guides about the library. It rarely started out with reading the Rust docs first, so I had to tell it to use the Rust docs. Therefore, I assume that providing quick access via tools to guides, docs, API references, and examples will spare me some roundtrips, time, and tokens and accelerate the process. I know that there are tools like context7, but it seems to be focused on JavaScript. At least for the Rust ecosystem, the docs are centralized and standardized and can even be used locally. A tool to access those quickly and find things in them would be highly beneficial.

Additionally, for coding projects, there are typical steps to follow to verify if the code is sound. It’s probably also best to provide specific tools or workflows to follow these steps exactly. Rules already help with this, though.

Last but not least, a natural next step would be for an agent to analyze its history periodically and provide new memory entries (rules) based on that to reduce repetition and common mistakes. For more details, see the Appendix: Most used prompts section, where I analyzed the top 10 most common prompts to help agents proactively develop useful rules.

If you’re curious about improvements, I’ve put together a wishlist for Claude Code that would make the agent even more useful.

Try It Yourself

If you want to experience the results of vibe-engineering firsthand:

# Clone and explore the codebase
git clone https://github.com/imgly/background-removal-rs.git
git checkout --tag v0.2.0

# Or install the CLI directly
cargo install --git https://github.com/imgly/background-removal-rs.git --tag v0.2.0

Remember: every line of code in this library was written by AI while I played the role of curator, architect, and quality guardian. Judge for yourself whether this new paradigm produces production-ready results.

Appendix

Appendix: Claude Code Wishlist

During the experiment, I encountered some issues whose resolution would improve the agent.

Project-scoped history (not global).
Project-scoped todos (not global).
Project-scoped implementation plans (not global).
Customizable /compact prompt or choose other compact strategies.
Improved handling of background processes.
History analytics with memory proposal.
Code-specific predefined Code and Validate workflows.
Allow forking of multiple agents from a single point in the session history to create multiple trials with the same intent and context.

Appendix: Most Used Prompts

Claude code stores the history under ~/.claude/history as json formats. I asked claude code to read them in and categorize the most used prompts.

#	Category	% Usage (Count)	Description / Examples
1	Other/Specific Instructions	19.5%	Technical specs, detailed requirements
e.g.: “Relax the success metrics”, “Use tensor data directly as alpha channel”
2	Questions	12.7%	Status checks, clarifications
e.g.: “What’s next?”, “How can I test it myself?”, “Do we have any mock implementations?“
3	Implementation Requests	11.1%	Direct build/create requests
e.g.: “Create a PRD for Rust port”, “Make ONNX Runtime injectable”
4	Simple Responses	10.0%	Short confirmations, approvals
e.g.: “ok”, “go for it”, specific technical choices
5	Continuation Commands	8.4%	Requests to proceed
e.g.: “go on” (59 times)
6	File Operations	8.3%	File management
e.g.: “Write this into PRD.md”, “Move packages into crates directory”
7	Context Summaries	8.3%	Session continuation due to context limits
e.g.: “This session is being continued from a previous conversation…“
8	Analysis/Review Requests	7.7%	Requests to analyze/review
e.g.: “Analyze my background removal project”, “Check the preprocessing”
9	Bug Fix/Issue Resolution	4.4%	Problem identification/fixes
e.g.: “You mix up release and debug”, “This is wrong”
10	Testing	3.8%	Running/validating tests
e.g.: “Test images are incorrect”, “Rerun comparison tests”

Appendix: Claude Code Settings

In real-world projects, the default timeout of two minutes is not enough. I bumped the timeouts to the maximum values to allow execution of unit tests, E2E tests, benchmarks, and long-running tasks.

BASH_DEFAULT_TIMEOUT_MS: Sets the default timeout (in milliseconds) for long-running bash commands.
BASH_MAX_TIMEOUT_MS: Specifies the maximum timeout (in milliseconds) that can be set for bash commands.
BASH_MAX_OUTPUT_LENGTH: Limits the maximum number of characters in bash outputs before they are truncated in the middle.
CLAUDE_BASH_MAINTAIN_PROJECT_WORKING_DIR: Ensures the working directory is reset to the original after each Bash command.

{
  "env": {
    "CLAUDE_BASH_MAINTAIN_PROJECT_WORKING_DIR": "true",
    "BASH_MAX_TIMEOUT_MS": "3600000",
    "BASH_DEFAULT_TIMEOUT_MS": "3600000",
    "MCP_TIMEOUT": "3600000",
    "MCP_TOOL_TIMEOUT": "3600000"
  }
}

Library Capabilities

High-performance Rust library for AI-powered background removal with hardware acceleration, built for
production scale and developer productivity.

Performance Highlights

Hardware acceleration: CUDA (NVIDIA), CoreML (Apple Silicon), CPU fallback
Sub-second processing on modern hardware (100-1200ms depending on image size)
Memory efficient with optimized threading and session reuse

Key Features

AI Models & Quality

Multiple state-of-the-art models (ISNet, BiRefNet)
FP16/FP32 precision variants for performance vs. quality
Portrait-optimized and general-purpose models
Custom model support via ONNX models from Hugging Face

Format and Color Profile Support

Input: JPEG, PNG, WebP, TIFF, BMP with ICC color profile preservation
Output: PNG (transparency), JPEG, WebP, TIFF, raw RGBA8

Integration Patterns

One-liner API for simple use cases
Session-based processing for batch operations
Stream processing from any AsyncRead source
CLI tool for standalone usage and pipeline integration

Architecture & Platforms

Dual Backend System

ONNX Runtime: Maximum performance, GPU acceleration
Tract: Pure Rust, zero external dependencies

Platform Support

macOS: Apple Silicon + Intel with CoreML acceleration
Linux/Windows: NVIDIA CUDA + CPU fallback

Developer Experience

Modern Rust Ecosystem

Async/await native support
Comprehensive documentation and examples
Zero-warning policy with extensive testing
Structured tracing for production observability

CLI Capabilities

Batch processing with recursive directory support
Model management (download, cache, clear)
Provider diagnostics and performance monitoring
Pipeline-friendly with stdin/stdout support

3,000+ creative professionals get early access to new features and updates. Don’t miss out, and subscribe to our newsletter.

IMG.LY x AI

Eray — Mon, 30 Jun 2025 13:18:14 GMT

Over the past two years, one question has consistently emerged in conversations with customers: “What about AI?”
While AI promises to disrupt many industries, it remains difficult to grasp how this technology will reshape creative workflows.

Our customers look to us for guidance: What is IMG.LY’s vision? How will our SDK help them harness this wave of innovation?
Last month, we took our first significant step by launching an initial suite of generative AI-powered features for our web SDK. We’ve embedded these tools deeply into photo, video, and design editing workflows and the response from customers and prospects was overwhelmingly positive.

And this is just the beginning. An immense opportunity lies ahead for both IMG.LY and our customers to drive transformation in the creative domain through our SDK. In this post, I want to share our vision for the future of creative tools powered by AI and our SDK.

Going forward, we’re focusing on three central goals:

1. Deep Integration of AI Capabilities into Editing Workflows

The pace of AI innovation continues to be remarkably fast. New models and configurations emerge almost daily, some as APIs, others open-sourced on platforms like Hugging Face. While some offer generalist features, others provide specialized, industry-specific capabilities, such as automatically obscuring license plates, blurring faces or replacing skies for property exteriors.

The true value of these AI tools emerges not in isolation, but when they’re seamlessly combined within existing workflows.

That’s why we built a plugin system for CE.SDK.
The CE.SDK plugin system has fundamentally changed how quickly we, and our customers, can integrate new capabilities. We’ve created a way to bring AI models and agents directly into connected workflows within the editor, making AI feel native rather than bolted on.

A recent example is our integration of OpenAI’s gpt-image-1 API, which we implemented in just a few days after its release. We used it to build a visual prompting workflow that takes into account all elements of a page, text, images, and annotations, to generate results based on the complete layout context.

These early successes are encouraging us to ship even faster and bring new features to our customers.

We’re now partnering with fal.ai to provide a nearly out-of-the-box experience to use any new generative AI model through their platform (more on this soon). Fal.ai excels at speed: when exciting new models emerge, they offer API access within days. This partnership ensures our users can quickly access new AI capabilities with minimal effort. More partnerships and integrations are on the horizon, bringing world-class AI APIs with intuitive interfaces directly into the editor.

2. Enabling AI Agents as Creative Collaborators

AI agents, like humans, leverage tools to accomplish tasks efficiently. CE.SDK occupies a unique position as a highly adaptable, multi-platform technology for editing various media types, including a fully documented headless version that’s perfect for programmatic control.

Thanks to CE.SDK’s fully documented API and headless architecture, AI agents can be created to navigate and operate the editor programmatically. This opens the door to agents that act as powerful scaffolders:

Generating initial designs and layouts based on brief descriptions
Automatically aligning content with brand guidelines
Transforming static designs into dynamic videos with a single command
Creating engaging short video content from prompts
Adapting existing designs to new formats and dimensions

Crucially, everything an AI agent creates remains fully editable by human users. You can refine, add nuance, and perfect the results through collaboration. AI provides the scaffolding, humans remain the tastemakers, adding the creative spark that makes designs truly exceptional.

3. AI-Powered SDK Configuration and Customization

AI agents are already actively involved in building, refining, and optimizing software tools, a trend that’s accelerating rapidly. This shift directly impacts developer experience and points to an exciting future where our SDK serves not just human developers, but AI agents as well.

To facilitate this interaction, we’ve already taken steps like making our documentation available in LLM-friendly formats. But this is just the beginning of our journey toward radically improving the experience for both developers and AI agents.

Our ultimate goal is conversational configuration. Imagine describing your requirements in plain language, and having an AI agent handle the rest:

Configuring the SDK’s visual aesthetics to match your brand
Setting up custom functionality and workflows
Integrating media libraries and plugins
Optimizing performance for specific use cases

This isn’t just about making development faster. It’s about democratizing access to powerful creative tools, allowing anyone to build sophisticated editing experiences, regardless of technical expertise.

Looking Forward

We believe that AI collaboration represents a transformative shift in creative technology, empowering users and developers to achieve extraordinary outcomes. At IMG.LY, we’re committed to being at the forefront of this exciting journey. Our vision extends beyond simply adding AI features, we’re reimagining how creative tools are built, configured, and used in an AI-augmented world.

Stay ahead with us: subscribe to our newsletter for exclusive updates.

AI-first Visual Editor using GPT-4o’s gpt-image-1 Model

Eray — Mon, 05 May 2025 20:58:07 GMT

What We Built

We integrated OpenAI’s new gpt-image-1 API (from GPT-4o) directly into our fully functional visual editor, CreativeEditor SDK (CE.SDK), enabling generation, editing, and refinement of images without ever leaving your creative workflow.

Open AI Editor Demo Page

From Simple Image Generation to Visual Prompting on a Canvas

Inside the editor, users can now:

Generate Images
Use prompts to generate images from scratch.

Generate Images from Visual Prompts
Turn full compositions (images, text, and annotations) into fresh visual content. Just select your page and let AI handle the rest, as shown in the video.

Reimagine Images & Text
Edit existing images and text with prompts to iterate faster and create variants.

Create Incredible Compositions
Combine generated and uploaded images into complex compositions.

Each step builds on the last, evolving from basic generation into true visual prompting powered by multiple input modes, all within one canvas. Check out the live demo here.

How We Built It

We built this integration using our CE.SDK and its flexible plugin system, designed from the ground up to support AI-first creative workflows.

This approach lets developers plug in any model or API (text, image, video, or audio), and run them all in one seamless editing flow. Whether you’re using OpenAI, Stability, or an in-house model, CE.SDK gives you the tools to bring it into the visual workflow natively.

🔗 Check out our AI Editor.
📘 Learn how to integrate AI into CE.SDK.

Why This Matters

Generative AI’s full potential isn’t unlocked by prompting alone, it’s unlocked when embedded into real-world creative workflows.

Designers, marketers, and content teams don’t just need outputs; they need control, iteration, and context. By bringing AI directly into the canvas where assets are created and edited, we turn generative models into tools for actual production, not just ideation.

This shift enables:

Creative work in context: No switching between ChatGPT and design tools.
Real-time augmentation: Prompt, edit, refine in place.
Scalable content generation: Automate localization, personalization, and variants.
Multimodal orchestration: Use visuals, layouts, and annotations as inputs.

It’s a step toward making multimodal AI usable for real design workflows, not just concept generation.

Integration & Feedback

This linked demo is rate-limited, if you would like to test more extensively or if you are interested in giving the AI editor a spin inside your own app, you can get started with our documentation.

We’d love your feedback, any thoughts, questions, and ideas are welcome!
Reach out to us.

3,000+ creative professionals gain early access to new features and updates. Don’t miss out, and subscribe to our newsletter.

OpenAI GPT-4o Image Generation (gpt-image-1) API: A Complete Guide for Creative Workflows for 2025

Jan — Mon, 28 Apr 2025 07:55:48 GMT

Update: AI-first Visual Editing

A day after the release of the gpt-image-1 API, we took it for a spin and integrated it into CreativeEditor SDK. Users can now generate images, create variants and use the canvas to compose visual prompts with our design editor. See it in action:

Open AI Editor Demo Page

Introduction

The release of OpenAI’s gpt-image-1 model signals a pivotal shift in the creative developer landscape, one that moves beyond static, one-shot image generation and toward a more dynamic, multimodal interaction model. Until recently, most image APIs followed a predictable pattern: submit a prompt, receive a finished image. The process was useful, but flat. What’s changing now is not just image quality or style fidelity, but the shape of the workflow itself. With gpt-image-1, built on the GPT-4o foundation, developers can start designing creative tools that feel conversational and iterative. This evolution invites a new kind of interface where prompting, tweaking, and refining happen inside the canvas, not outside of it.

For teams building creative editing experience into their app, this moment coincides with the release of IMG.LY’s AI Editor SDK, a powerful, fully integrated toolkit designed for generative workflows. The SDK is already equipped to support interactive image generation, contextual editing, and multimodal inputs, and you can try it today through this live demo.

This guide is a comprehensive introduction to the gpt-image-1 API, but it also goes further. It’s not just about wiring up an endpoint, it’s about rethinking what image generation means in a user-centric product.

From prompt handling to interactive iteration, we’ll walk through how to design creative cycles, not just outputs. This guide explores how to make that shift, how to go from generating images to integrating gpt-image-1 into real creative cycles, where AI becomes a tool that bends to user intent, not the other way around.

Overview of `gpt-image-1`

OpenAI’s gpt-image-1 model, released in April 2025, is the latest evolution in the company’s generative image lineup and marks a turning point in how developers approach visual creation inside applications. Built on the same multimodal foundation as GPT-4o, this model allows applications to move beyond one-shot static generation and instead build toward more conversational, iterative image workflows.

Model Architecture and Capabilities

gpt-image-1 is rooted in GPT-4o’s ability to understand and generate across modalities. It is designed to produce high-resolution images (up to 4096×4096 pixels) based on natural language prompts. The model handles complex scenes with more fidelity than previous iterations and provides improved consistency in how it interprets detailed descriptions. This is particularly relevant for tools that need reliability when turning prompt inputs into design elements.

Parameter Control

Developers working with gpt-image-1 have access to a streamlined set of parameters, here is a subset of the most important ones:

prompt: The primary text input describing the desired image.
size: Choose between “1024x1024”, “1024x1536” (portrait), “1536x1024” (landscape), or “auto” (default, based on prompt).
n: Number of images to generate (default is 1).
response_format: Always returns b64_json. URL outputs are not supported.

Unlike DALL·E 3, gpt-image-1 does not accept style modifiers or quality settings. It is designed for straightforward, high-fidelity image creation driven purely by the text prompt and size selection.

Full documentation of these options is available via OpenAI’s official guide.

Style and Use Case Alignment

By supporting a wide range of stylistic templates, gpt-image-1 positions itself as a flexible backend for everything from marketing collateral to storyboarding tools. The output can be tailored to suit technical illustrations, concept art, or even photorealistic renderings, allowing developers to map visual outputs more directly to brand or product requirements.

Limitations and Future Direction

As of April 2025, gpt-image-1 supports only one image per request and does not offer fine-grained image editing or inpainting. However, its tight coupling with GPT-4o suggests that future iterations may embrace persistent context, conversational refinement, or even integrated image-plus-text exchanges within the same session. For developers building editors or multimodal workflows, the current model lays a strong foundation for these future capabilities.

API Setup and Usage

2.1 Get Access

To start using gpt-image-1, developers must first register for access via the OpenAI platform at platform.openai.com. Access requires an API key, which is tied to your OpenAI account and associated usage limits based on your billing tier. Be sure to confirm that your account is approved for image generation, as availability may differ by region and subscription level. Once authenticated, keys can be created in your dashboard and stored securely in your server or development environment.

2.2 First Image Generation (Node.js Example)

The image generation API for gpt-image-1 can be used directly via OpenAI’s official Node.js client. Below is a complete example showing how to send a prompt and receive an image URL in response:

import OpenAI from 'openai';
import fs from 'fs';

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY, // make sure this is securely set
});

async function generateImage() {
  try {
    const prompt = `
    A studio ghibli style illustration of a cyberpunk girl holding a butterfly on her finger.
    `;

    const result = await openai.images.generate({
      model: 'gpt-image-1',
      prompt,
      size: '1024x1024', // or "1024x1536", "1536x1024", or "auto"
    });

    const image_base64 = result.data[0].b64_json;
    const image_bytes = Buffer.from(image_base64, 'base64');
    fs.writeFileSync('butterfly.png', image_bytes);
    console.log('Image saved as butterfly.png');
  } catch (err) {
    console.error('Error generating image:', err);
  }
}

generateImage();

Remember that all outputs from gpt-image-1 are delivered as base64-encoded JSON. Developers should decode this data for display, storage, or further processing within their applications. For complete parameter options and examples, consult the OpenAI Images API guide.

Integrating with CE.SDK

Embedding gpt-image-1 into a creative editor like CE.SDK is about more than just piping an image into a canvas. It reshapes how users interact with content creation, bridging manual design work and AI-driven generation within the same editing environment. Rather than operating as a standalone prompt generator, gpt-image-1 becomes a continuous creative partner inside your editor. For in in-depth technical guide on how to integrate gpt-image-1 stay tuned for our upcoming tutorial, sign up to our newsletter to be notified when it goes live.

Embedding Image Generation in a Creative Editing Workflow

The natural entry point for gpt-image-1 inside CE.SDK is through a dual-mode experience: offering users the option to start either from scratch or from existing context. In “from scratch” mode, a user might open a blank scene and initiate an image generation by writing a prompt for example, “Create a vibrant festival scene at sunset.” The result appears directly on the canvas, immediately editable like any other design element.

Where gpt-image-1 shows its real potential is in “in-context editing.” Here, users interact with existing content (a background, a product shot, or a decorative element) and trigger AI enhancements based on that visual context. A user might select an image of a bird, as in the example below and ask for variants, initiate a background swap, or request a change like adding more birds in a conversational interface embedded in the editor. Because CE.SDK treats generated images as first-class canvas elements, context such as positioning, layering, and cropping is preserved throughout the process.

Let’s see what this might look like in practice. We positioned an image of a single bird on our canvas, opening the AI context menu we can now manipulate that image in place using the OpenAI API:

We edit the image and prompt the API to add more birds:

We see that the model correctly identified the type of bird in the picture (seagull) and filled it in with a swarm of flying seagulls.

We can now continue to work with the image, overlaying filters, changing the texture, cropping etc.

Switching Between Manual Edits and AI-Powered Enhancements

A critical design principle when integrating gpt-image-1 is giving users freedom to toggle between manual edits and AI suggestions. Manual edits should always remain possible after generation, e.g. cropping, masking, compositing while users can also seamlessly prompt gpt-image-1 for additional changes without losing prior work. Think of variant generation as a branch: a user picks a generated image and creates “forks” by asking for alternate styles, different lighting, or new thematic elements.

In this setup, the generated image serves as a stable node in the creative graph, while edits and regenerations can attach contextually. This workflow minimizes user frustration by avoiding the “start over” penalty typical of isolated generation APIs. It also opens up more complex creative behaviors, like blending user-drawn sketches with AI-augmented refinements, or iteratively developing an asset library around a consistent visual theme.

An upcoming in-depth tutorial will walk through implementing this multimodal workflow step-by-step, but the key takeaway is that gpt-image-1 shines brightest when it is embedded into a creative loop, not treated as a black-box generator, but as an interactive, iterative design companion.

Prompt Engineering Tips

One of the most overlooked but critical factors in successful image generation is prompt design. With gpt-image-1, prompt engineering isn’t just about describing an image. It’s about steering the model toward intent, tone, composition, and usability. Because the model is capable of rendering complex scenes and a wide range of styles, thoughtful phrasing and contextual hints can dramatically affect the outcome.

Writing for Visual Intent

Start by clarifying what the image is supposed to communicate. Are you looking for atmosphere, action, product detail, or narrative clarity? A prompt like “a city skyline at night” is a starting point, but it leaves too much to chance. Adding elements like “view from a rooftop bar, with glowing signage and overcast haze” gives the model anchors for both composition and mood.

Leveraging Artistic Language

You can further refine outputs by referencing mediums or artistic schools. Prompts that include terms like “in watercolor style,” “oil painting,” ”80s anime aesthetic,” or “studio photography” help the model lock onto a particular visual identity. These cues not only improve stylistic fidelity but also align the output with specific brand or genre expectations, which is especially important for products with a defined look and feel.

Creating Consistency in Branded Outputs

When generating a set of related images, such as social media creatives, campaign assets, or UI visuals, consistency becomes more important than variety. To achieve this, structure prompts with repeatable patterns and include brand elements such as color palettes, motifs, or reference characters. While gpt-image-1 doesn’t yet support persistent memory across requests, consistency can be enforced by prompting with the same style terms, layout descriptions, and constraints. Teams working within CE.SDK can even pair prompt templates with locked canvas layers to preserve composition between generations.

Ultimately, good prompt engineering is not about verbosity but about clarity and constraint. It’s less like writing poetry and more like drafting a product spec. The best prompts are focused, directive, and give the model just enough creative freedom within clear boundaries. However, effective prompting should not burden the user. In practice, the interface should abstract most of the complexity away. Users can be guided toward better outputs through simple UI choices (selecting predefined styles, choosing themes, or adjusting mood settings) while the system dynamically enhances and augments their input behind the scenes. By managing the technical depth invisibly, you enable a creative process that feels intuitive and powerful without ever making prompt engineering the center of the user experience.

Real-World Use Cases

The versatility of gpt-image-1 makes it especially impactful across a variety of industries where visual content creation is either a core product feature or a major operational need. Beyond isolated image generation, the model supports workflows that demand contextual awareness, brand consistency, and iterative refinement, key ingredients for modern digital products.

Web-to-Print

In web-to-print applications, customers expect to customize marketing materials, event invitations, signage, or packaging with minimal friction. By integrating gpt-image-1, developers can offer template-driven personalization where users simply select a theme or enter a few keywords, and receive ready-to-edit visual assets. Combined with CE.SDK’s layout and editing capabilities, this enables a highly interactive experience where generated backgrounds, graphical elements, or themed illustrations can be dynamically placed into editable templates.

Marketing teams rely on high-frequency content creation, often needing visually consistent, campaign-specific assets. gpt-image-1 can assist by automating the generation of background scenes, promotional visuals, and thematic graphics based on campaign briefs. Brands can define style presets aligned with their visual identity, making it easy for marketing teams to produce “on-brand” assets without heavy design overhead. Integrating image generation directly into campaign builders or social scheduling tools amplifies speed without sacrificing quality.

Digital Asset Management (DAM)

Asset libraries often suffer from gaps: missing variants, seasonal versions, or content tailored to different demographics. DAM systems can integrate gpt-image-1 to extend asset catalogs dynamically. Instead of manually commissioning variations, users can generate alternative backgrounds, localize visuals with region-specific elements, or adjust brand visuals for different markets, all from a single master file. With CE.SDK handling structured editing, teams maintain asset consistency while boosting creative flexibility.

E-Commerce

Product visualization remains a huge challenge in e-commerce, especially for smaller retailers. gpt-image-1 can be used to automatically create product lifestyle imagery, context backgrounds, or thematic campaigns without expensive photo shoots. For example, a single shoe photograph can be placed into a generated “urban,” “sporty,” or “luxury” background, customized according to target audiences. When tightly integrated into e-commerce platforms, this enables faster product launches, A/B tested visuals, and localized campaigns at scale.

E-Learning

Educational platforms can harness gpt-image-1 to generate explanatory diagrams, thematic illustrations, or scene-based visual storytelling assets. Instead of relying solely on static stock imagery, teachers, course designers, or even learners themselves can prompt the generation of custom visuals aligned with the curriculum. When embedded into authoring tools, this approach accelerates content creation and enables more engaging, visually enriched learning experiences tailored to specific topics and age groups.

Cost Optimization

While gpt-image-1 opens up impressive creative possibilities, it also introduces new cost considerations that developers and product teams must plan for carefully. Since image generation typically incurs higher API costs than text-based operations, structuring workflows efficiently becomes critical, especially at scale.

Balancing Price, Quality, and Resolution

The cost of generating an image with gpt-image-1 depends significantly on both the requested resolution and the selected quality setting. Higher resolutions like 4096×4096 produce sharper, more detailed results, but they also consume more compute resources-and therefore cost more. For many use cases, especially for previews, lower resolutions such as 1024×1024 or 2048×2048 strike an excellent balance between visual fidelity and API efficiency. Reserving the highest quality settings for final exports or premium workflows can help manage overall spend without compromising user experience.

Image Reuse and Smart Upscaling

One practical cost-saving approach is to design workflows that encourage image reuse. Instead of regenerating similar images for every small variation, applications can create high-quality master images and allow users to crop, edit, or layer additional design elements dynamically. Integrating smart upscaling techniques-for instance, using specialized image enhancement libraries after initial generation-also allows teams to work with smaller base images without sacrificing end-user quality.

Rate Limits and Batching Strategies

Every call to gpt-image-1 counts toward your usage quota, and OpenAI imposes rate limits depending on account tier. To optimize performance and cost, it’s helpful to batch generation requests thoughtfully where possible-for instance, combining multiple prompts into structured queues or allowing users to preview low-res draft versions before finalizing a high-res render. Building this logic into your app’s generation flow not only controls expenses but also improves perceived responsiveness, an important UX factor for creative applications.

By considering cost optimization as an early design constraint rather than a late-stage patch, developers can build scalable, sustainable creative tools powered by gpt-image-1.

Bonus: Starter Kit Repo

We are currently in the process of integrating the new GPT-4o-powered gpt-image-1 model into CE.SDK. As part of this effort, we are preparing a comprehensive Starter Kit will showcase a complete with CE.SDK integration, real-time prompt input, image generation workflows, and best practices for building an AI-powered creative editor.

Both a public GitHub repository and a live demo will be made available soon. If you want to be notified when the Starter Kit launches, you can subscribe to updates here.

This Starter Kit is designed to help developers move beyond simple image generation into building full creative cycles, where users can generate, edit, refine, and remix visuals seamlessly inside the editor.

FAQs

Choosing to work with gpt-image-1 raises a number of practical and strategic questions. Below, we address the most common topics for teams evaluating the model for integration into creative workflows.

How is `gpt-image-1` different from DALL·E 3?

While DALL·E 3 and gpt-image-1 both translate text prompts into images, the underlying architecture and integration paths are different. gpt-image-1 is built on GPT-4o’s multimodal framework, making it better suited for future conversational and iterative workflows. It also offers support for a wider range of styles, higher resolutions up to 4096×4096 pixels, and is positioned for deeper integration into dynamic user experiences rather than one-off generation tasks.

Can you fine-tune or train `gpt-image-1`?

As of April 2025, OpenAI does not allow fine-tuning of gpt-image-1. The model is optimized for broad creative use cases out of the box. Developers seeking more control typically customize the user-facing prompt engineering or combine outputs with structured editing tools like CE.SDK to achieve brand or project-specific consistency.

Is offline support available?

Currently, gpt-image-1 requires access to OpenAI’s cloud APIs. There is no offline inference mode or local deployment option. Teams requiring strict data residency, offline workflows, or private model hosting should consider hybrid architectures where images are generated securely via backend services and then edited locally using embedded tools like CE.SDK.

What about copyright and licensing?

Images generated by gpt-image-1 can be used commercially according to OpenAI’s usage policies, but developers are encouraged to review the latest terms. Outputs are not directly copyrighted by OpenAI or the user, and responsibility for ensuring compliance with branding, likeness, or content standards typically falls on the developer or platform operator. When deploying generation features to end-users, it is good practice to provide clear terms of use and, if needed, additional moderation or review layers.

By addressing these considerations early, teams can integrate gpt-image-1 more effectively and responsibly into creative products and workflows.

Conclusion

gpt-image-1 offers developers a significant opportunity to rethink what image generation can mean inside creative applications. It is not simply a tool for producing pictures on command, but a foundation for building interactive, iterative design workflows where users stay in control of the creative process. When combined with CE.SDK, it becomes even easier to move from static outputs to living, editable canvases that support real-world design needs. As we continue to integrate GPT-4o capabilities, the next wave of creative tooling will be about more than prompting images-it will be about shaping truly collaborative creative environments. Now is the time to start experimenting, iterating, and reimagining the user experience around this new generation of multimodal AI.

How OpenAI's Upcoming GPT-4o Image Generation API Will Change Creative Workflows

Jan — Mon, 14 Apr 2025 10:51:53 GMT

If you’ve been working with image-generation APIs over the past year, you’ve probably gotten used to a certain flow: send a prompt, wait a few seconds, and get a flat image back. It’s a one-shot deal. Useful? Definitely. But not exactly interactive. That’s what will change with OpenAI’s upcoming GPT-4o image-generation capabilities.
IMG.LY, which recently released a suite of AI features for its design editor, is eagerly awaiting the release to expand how users can interact with AI-driven creativity even further.

Update: AI-first Visual Editing

A day after the release of the gpt-image-1 API, we put the UX principles outlined in this post into practice and integrated it into CreativeEditor SDK. Users can now generate images, create variants and use the canvas to compose visual prompts with our design editor. See it in action:

Open AI Editor Demo Page

GPT-4o: Beyond the Prompt-to-Image Pipeline

GPT-4o isn’t just another version of DALL·E. It represents a shift in how developers will integrate AI into creative applications. While DALL·E 3 is powerful it is also somewhat siloed (you send a prompt, you get an image), GPT-4o looks like it will be part of a much more dynamic, conversational model one that accepts both text and image inputs, and could soon generate visual content in context, on the fly, and as part of a back-and-forth user interaction.

If you’ve used ChatGPT recently, you’ve already seen glimpses of this. You can drop an image into the chat, ask GPT to describe or edit it, and get a response that feels fluid and visual. Developers should expect the API version to follow a similar pattern. It likely won’t just be a /generate-image endpoint. Instead, we may be looking at an extension of the chat/completions endpoint that handles multimodal messages. That changes the way you integrate this capability into your application. Rather than simply placing an image generation step in your pipeline, you will have to build your app’s UX around this new user flow. This comes with its own set of unique challenges.

Rethinking the Interface: Prompting as a Conversation

So what does this mean if you’re planning to integrate multi-modal image generation into your own product? For starters, you’ll probably need to rethink how users initiate and refine prompts. In the DALL·E flow, you might offer a text box with a few style dropdowns and call it a day. But in a GPT-4o world, your UI needs to support image inputs, persistent context, and dynamic editing, image gen becomes more like a conversation than a command.

This is where the rubber meets the road. The tools that will benefit most from GPT-4o aren’t static generators but interactive editors. Think collaborative design apps, video editors with generative overlays, or product customizers that let users sketch or upload a photo and then iterate with AI. Put differently, the model output isn’t the endpoint but rather a checkpoint in the creation process.

A Typical Iteration Cycle in a Multimodal Workflow

Here’s a rough sketch of a workflow we might be seeing more of: The user starts with a prompt and an image, maybe a rough sketch or collage created inside an editor, a product photo, or a UI frame. GPT-4o returns a generated image based on that input. The user then edits or annotates the result, maybe adds new prompt text for refinement, and resubmits that combination to further develop the output. This cycle might loop several times: generate, tweak, refine, regenerate.

That’s a fundamentally different interaction model from past AI tooling. It’s less about one-off generation and more about a guided creative journey, where the user is in dialogue with the model. The result: better alignment with the original intent, more control, and more usable creative outputs.

There is an additional, more subjective benefit to this kind of workflow: it gives the user a sense of autonomy again; they are back in the driver’s seat and less at the whim of an inscrutable machine. In many contexts, that makes a difference. Most notably, as we discussed in our white paper on print personalization, the psychological benefit of personalization lies to a large extent in the investment, the sense of ownership that comes about when you create something. “Make it yours” is the common tagline attached to personalization campaigns in e-commerce. That only works if the user exerts more control over the output than iterating over a set of prompts.

The most pithy encapsulation of this paradigm that I have heard is Humans on top, AI on tap.

Persistent Elements and Visual Consistency

One particularly interesting frontier here is character and object persistence. If a user defines a character early in the workflow, either via prompt, image, or a combination, they’ll increasingly expect that character to appear consistently across assets. Think of it as visual continuity, whether you’re generating scenes in a story, slides in a deck, or frames in a video.

If the user of a creative marketing cloud creates a campaign avatar or mascot, that character needs to be consistent within and across campaigns.

Being able to reference earlier outputs, prompts, or style cues gives the user control over not just individual assets but the whole arc of the design narrative. GPT-4o’s ability to maintain that continuity is a game-changer for workflows that involve storytelling, brand identity, or serialized design work.

What to Expect from the API

Technically, if GPT-4o follows OpenAI’s recent design philosophy, you can expect a JSON-based API with a messages array, where content can include both text and image_url types. The output will likely be returned either as an image URL hosted by OpenAI or as base64-encoded image data, depending on the format you request.

That structure plays nicely with modern JavaScript front-end frameworks. React, Svelte, and Vue are all well-suited to async generation flows with visual previews. If you’re already using tools like Zustand or Jotai for local state or something like tRPC or GraphQL for structured calls, you’re in a good position to layer GPT-4o in without breaking the flow.

Trade-offs and Technical Considerations

There are trade-offs, of course. GPT-4o will probably cost more per call than a standard DALL·E 2 or 3 generation. Its latency is still an open question, and the multimodal input support will likely require more thoughtful UX decisions. What happens when a user drops an image and wants to undo just part of the generation? Where do you store prompt context for edits? How do you communicate what’s editable and what’s not?

This is where design and engineering need to work together. You’ll want to build an interface that makes AI feel like a creative partner, not just a backend service. That might mean giving users a visual prompt history or allowing partial re-generations of specific canvas elements. You’ll need sensible fallback states. What happens when generation fails or the result isn’t what the user wanted?

Where IMG.LY’s CE.SDK Fits In

We have already given the questions raised above some serious thought, and most of the complexities introduced by this new workflow are the table stakes for the Creative Editor. So, if you’ve already integrated IMG.LY’s CE.SDK, we have taken care of most of these problems, and you can seamlessly integrate with any AI model. We are actively working on an off-the-shelf integration of the GPT-4o image model once its public API launches.

In general, you can treat GPT-4o’s image outputs as just another layer in the editing canvas, positioned, styled, cropped, and ultimately editable in the same environment as everything else. That’s the real power of multimodal workflows: not just generating but integrating. And once GPT-4o’s API goes live, you’ll want your infrastructure ready to slot it in with minimal friction.

The Loop: Prompt, Generate, Refine

The era of single-shot generation is winding down. What’s coming next is a loop: edit, prompt, generate, refine, repeat. And this loop doesn’t just belong in the backend, it needs to live in the UI, in a way that invites user input, creativity, and correction.

We’ll be publishing more on how this integrates into IMG.LY’s upcoming AI workflows soon. Expect tools that don’t just generate visuals but help teams and individuals work through ideas in real time. Because especially as AI gets more potent, it needs humans on top.

3,000+ creative professionals gain early access to new features and updates. Don’t miss out, and subscribe to our newsletter.