Skip to main content
Your app can use AI models to generate text, images, speech and video, and to transcribe audio. You don’t need an account or API key with any AI provider. Requests are paid for with your organization’s Newly credits.
This is about AI inside the app you build. It doesn’t change the AI that builds your app. Newly chooses the model behind Newly Agent.

​
Add an AI feature

1

Open the AI models menu

In the prompt box, click +, then AI models. The menu says “Choose what your app can create”.
2

Pick one or more models

Filter by Text, Images, Audio or Video, or search by model or provider. Click a model to add it to your message. To add another, open the menu again. Models marked “Unavailable” can’t be picked right now. The list can change over time.
3

Describe the feature

Say what the feature should do. For example: “Add a button that turns the user’s photo into a watercolor painting” or “Let users ask questions about their saved notes.”
The agent turns the models on for your dev backend, writes the server code that calls them and connects your app’s screens to it. You can try the feature in your preview. Turning models on needs the project’s owner or a project Admin. If you aren’t one, the agent can’t turn them on and gets “An owner or admin must manage backend AI access.”

​
How it works

  • Newly Backend with sign-in is required. Your app calls AI models through its own Newly Backend, on behalf of a signed-in user. See Sign-in for your app.
  • No keys in your app. The access your backend uses to reach AI models is added when the backend is deployed. It never goes into the app itself.
  • Each environment is separate. Models are turned on separately for dev and production.

​
Credits

Every AI request your app makes uses your organization’s credits, in dev and in production. That includes requests from your app’s users, not only your own testing. It’s the same credit balance you use for the agent. There’s no separate AI wallet.
  • Before a request runs, Newly sets aside the most it can cost. You’re never charged more than that amount, and you’re charged for what the request actually used.
  • Failed requests don’t use credits.
  • Requests cancelled before they start don’t use credits. Once a request has started, it can’t be cancelled.
Larger or longer outputs can cost more, for example longer videos or longer text. See How credits work.
When your app is live, your users’ activity spends your credits. Keep an eye on your balance, especially after a launch. If credits run out, AI features in your app stop working until you add more.

​
Limits

These limits apply to each project environment, with dev and production counted separately: If your app needs to keep a generated image, audio clip or video longer than 7 days, ask the agent to save a copy in your app’s storage.

​
Messages your app may receive

Ask the agent to show a friendly message in your app for each of these.

​
Turn on AI models for production

The agent can only turn models on for dev. For your released app, turn them on for production yourself with the newly CLI. You need to be the project’s owner or a project Admin. Otherwise you’ll see “An owner or admin must manage backend AI access.”
1

Set up the newly CLI

Install the tool, sign in and pull your project. Follow the steps in Production and secrets.
2

Find the model ids

From your project folder, run:
3

Turn the models on

List every model your app uses in dev, separated by commas. The command replaces the production list, so include all of them each time:
4

Deploy the backend to production

The deploy connects your production backend to the models.

How credits work

What uses credits and what doesn’t.

Production and secrets

Deploy your backend to production.