Amplify Backend Gen1 → Gen2 Migration (Beta)

[iliapolo]

Amplify team has launched a beta version of a flavor of the Gen1 CLI that adds support for migrating Gen1 backend environments to Gen2.

Caution

The migration tool discussed here is in early stages of development and SHOULD NOT be
executed on any production or mission critical environments. Only run it for testing
purposes on environments you can afford to delete.

Your feedback will significantly help the team improve and stabilize the tool. We welcome and encourage any feedback on the migration process:

• Success stories
• Failure stories
• General Questions
• Anything you want

Get started by following our migration guide. We are eager to hear what you think and incorporate your feedback in the upcoming versions!

Thanks.

[iliapolo]

Heads up folks we identified a code generation bug: #14492

It pretty elusive at the moment and doesn't happen on every app. If this happens to you, please let us know and share your setup.

Thanks!

[iliapolo]

Fixed

[jamilsaadeh97]

Is there any reason why Flutter is not supported? Or is it untested yet

[iliapolo]

@jamilsaadeh97

if I went ahead and tried the tool, would those features be overwritten/removed or can I manually reference or add them in the gen 2 code without losing them?

Analytics and notifications would not be code generated so your Gen2 environment will be missing those. You can manually add them in using some fairly complex CDK code - but since they also need special handling in the refactor phase i'm afraid you won't be able to complete migration right now.

The other features you mentioned (auth trigger, guess access) should be possible to compensate with some manual editing, but I can't say exactly what that would look like right now.

Im happy to share more insights into my current stack if you want.

Anything you can share will be really helpful. If you have the exact CLI commands you ran to setup your app that would be really great because we could reproduce your setup.

Im also intersted in:

• Does your flutter code live in the same repo (and folder) as your amplify directory?
• How are you deploying your Gen1 environment?
• How are you consuming the amplifyconfiguration.json file in your flutter code?

Thanks!

[jamilsaadeh97]

Analytics and notifications would not be code generated so your Gen2 environment will be missing those. You can manually add them in using some fairly complex CDK code - but since they also need special handling in the refactor phase i'm afraid you won't be able to complete migration right now.

So I can't use this to reference the existing analytics and notifications resources without losing them when deploying the newly migrated gen2 app?

import { defineBackend } from "@aws-amplify/backend"

const backend = defineBackend({})

backend.addOutput({
  analytics: {
    amazon_pinpoint: {
      aws_region: "<your-aws-region>",
      app_id: "<your-pinpoint-app-id>",
    },
  },
})

This is coming from the docs

If you have the exact CLI commands you ran to setup your app that would be really great because we could reproduce your setup.

I will try to provide the exact CLI commands for my setup. This Amplify app is fairly big and grew a lot over the last 4-5 years so I will need to recreate the gen 1 app from scratch and note the commands. Unless there's an easier way I don't know about?

Does your flutter code live in the same repo (and folder) as your amplify directory?

It used to, but I changed it to be an isolated repo for the backend, and multiple frontends (each living alone) connected to the same backend. I pull the changes with amplify pull --appId xxx --envName xxx in each frontend repo when needed.

How are you deploying your Gen1 environment?

If you mean my flutter app, I deploy it to the App Store and Play Store, not hosted using Amplify Hosting since we don't have a web app yet, just a mobile app. And if you mean the Amplify environment, manually using amplify push

How are you consuming the amplifyconfiguration.json file in your flutter code

Since I specified in the cli that it's a flutter app, the gen 1 cli generates an amplifyconfiguration.dart and not a .json to be consumed by the Amplify Flutter library using Amplify.configure().

Please let me know if I missed some info to provide, will be more than happy to provide more in order for you guys to gain as much insights as possible.

[jamilsaadeh97]

To give a general idea about the setup:

Auth

• Login using email or phone number
• Guest access enabled
• Required attributes are first name, last name, phone number, email, birthdate
• We have one group called "Partners"
• Lambda triggers: Pre signup, Post Confirmation, Custom Message
• No Admin Apis

Api

• Default auth mode: Cognito User Pool
• Additional auth mode: IAM
• DynamoDB lambda triggers on different models

Storage

• Content
• Restricted access to Auth(read, create)/Guest(read) users
• Lambda trigger

Function

Multiple lambda functions with different access to Cognito, DynamoDB, S3, Pinpoint

And finally Pinpoint analytics and notifications.

This is in a nutshell, it should give a pretty close setup.

[iliapolo]

So I can't use this to reference the existing analytics and notifications resources without losing them when deploying the newly migrated gen2 app?

import { defineBackend } from "@aws-amplify/backend"

const backend = defineBackend({})

backend.addOutput({
  analytics: {
    amazon_pinpoint: {
       aws_region: "<your-aws-region>",
       app_id: "<your-pinpoint-app-id>",
    },
  },
})

You could. But there are a few issues with that:

• Your pinpoint resources won't be managed by the new Gen2 environment (i.e the underlying CloudFormation stack) - so you won't be able to evolve them going forward via the Gen2 code.
• You mentioned your functions access pinpoint - so you'd need to manually edit the generated Gen2 code to configure that access (permissions and env variables).
• Eventually you'd also want to decommission your old Gen1 environment without deleting these pinpoint resources.

For all of this to work, the migration tool needs to code generate the right CDK code and perform a CloudFormation refactor so that these resources will be owned by the Gen2 stack. This is currently not supported.

So yeah you could do that for now for testing purposes but its not a long term solution. Unless your pinpoint resources are managed externally from your Gen1 environment - are they?

And if you mean the Amplify environment, manually using amplify push

Yeap thats what I meant. So after you migrate to Gen2 - do you have an idea on how you'd like to deploy the app?

• Will you leverage Fullstack branch deployments even though you don't have a webapp?
• Or will you run npx ampx pipeline-deploy --branch <your-branch> --app-id <your-app-id> from your own external pipeline?

This is in a nutshell, it should give a pretty close setup.

Thank you! This helps a lot.

[jamilsaadeh97]

So yeah you could do that for now for testing purposes but its not a long term solution. Unless your pinpoint resources are managed externally from your Gen1 environment - are they?

You're right, the issues you mentioned are valid for us. No they are not managed externally.

Yeap thats what I meant. So after you migrate to Gen2 - do you have an idea on how you'd like to deploy the app?
Will you leverage Fullstack branch deployments even though you don't have a webapp?
Or will you run npx ampx pipeline-deploy --branch <your-branch> --app-id <your-app-id> from your own external pipeline?

I've played around with Gen 2 a while ago, and I think at first I will be deploying using the command. In the future I might leverage the fullstack branch deployment. I also saw that it's possible with a dummy frontend (empty index.js). This way I can commit and push and the backend will be deployed automatically. In the future, when we will develop a web app, the whole thing will be ready and we just replace the frontend part.

Thank you!

[OperationalFallacy]

I didn't see anything mentioned about DataStore alternatives? Imho, that's a big deal for the app functionality.

I would also try to have something like a high level map of features of gen1 to gen2 from admin perspective, otherwise it's easy to get lost how things function or what's their purpose, or where things managed.

I wrote about migration path a while back from that point of view
https://thecloudtimes.substack.com/p/aws-amplify-migration-from-gen1-part

[OperationalFallacy]

clear mapping of how the migration tool will choose to implement each Gen1 feature in Gen2?

yes, this. I think you already maintain nice list on the doc website on features parity.

In addition to that, what I meant "admin perspective" is how development and deployments flows change. Would allow teams plan the migration better.

For example, sharing outputs files between be and front-end was a big friction point for me on a mono repo setup, had to figure out how to package and share it properly. The UI/forms flows changed, too.

[iliapolo]

For example, sharing outputs files between be and front-end was a big friction point for me on a mono repo setup

Can you elaborate on this?

[OperationalFallacy]

Outputs is backend generated, from deployed resources and part of the deployment workflow. The backend code will need to consume the file to access deployed properties (in Lambda for example) in the repos that beyond toy projects. That's a first self-feeding loop.

In addition, the outputs file is also directly consumed by front-end. Also, outputs file is an integration point for the custom props.

In a repo setup, with backend and front-end separated, workflow to share output updates become none trivial. And is very different from gen1.

[iliapolo]

In Gen2 you can generate the amplify_outputs.json file using this command:

npx ampx generate outputs --app-id <app-id> --branch <branch-name>

This file is the Gen2 equivalent of the Gen1 amplifyconfiguration.json and client libraries can work with both. Is that not sufficient to provide a similar workflow to Gen1?

The backend code will need to consume the file to access deployed properties (in Lambda for example) in the repos that beyond toy projects

Do you mean application backend code? or other infra code? If you can give a concrete example that would be great. Thanks!

[OperationalFallacy]

In backend I do something like below.
It lets me decouple Lambda code from the Amplify helpers.
So I can override service layer based on the outputs config with local creds loaders, override amplify client for unit testing, guard setup so envs doesn't get mixed up

import { parseAmplifyConfig } from "@aws-amplify/core/internals/utils";
import { fromNodeProviderChain } from "@aws-sdk/credential-providers";
import { Amplify, type ResourcesConfig } from "aws-amplify";
import { generateClient } from "aws-amplify/api";
import { Effect, Schema as S } from "effect";
import outputs from "../../../amplify_outputs.json";
import type { Schema } from "../../../data/resource";

export const AmplifyOutputsSchema = S.Struct({
	custom: S.Struct({
		env: S.Struct({
			name: S.String,
		}),
		events: S.Struct({
			url: S.String,
		}),
	}),
});

export type AmplifyRuntimeService = {
	readonly client: ReturnType<typeof generateClient<Schema>>;
	readonly eventsEndpoint: string;
};

export class AmplifyRuntime extends Effect.Service<AmplifyRuntime>()(
	"AmplifyRuntime",
	{
		effect: Effect.gen(function* () {
			const decodedOutputs =
				yield* S.decodeUnknown(AmplifyOutputsSchema)(outputs);
			const amplifyConfig: ResourcesConfig = parseAmplifyConfig(outputs);
			const credentialsProvider = fromNodeProviderChain();

			Amplify.configure(amplifyConfig, {
				Auth: {
					credentialsProvider: {
						getCredentialsAndIdentityId: async () => ({
							credentials: await credentialsProvider(),
						}),
						clearCredentialsAndIdentityId: () => {},
					},
				},
			});

			yield* Effect.logDebug("Amplify config loaded");

			const eventsEndpoint = decodedOutputs.custom.events.url;
			const runtimeService: AmplifyRuntimeService = {
				client: generateClient<Schema>({ authMode: "iam" }),
				eventsEndpoint,
			};

			return runtimeService;
		}),
		dependencies: [],
	},
) {}

export const AmplifyRuntimeLive = AmplifyRuntime.Default;

The outputs file gets compile packaged, so front-end can import it from the same location and load it in a similar way with overrides it needs.

[hisham]

Thanks for the update. I haven't tested yet, but off the top of my head we use lambda layers heavily (well we just have 1 layer but it's shared with multiple lambdas). So I think we can't test this migration until lambda layers is supported in gen 2?