Skip to content

optimize Cedar migration: replace N+1 queries with single query #1656

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gugu merged 1 commit into from
Mar 10, 2026
Merged

optimize Cedar migration: replace N+1 queries with single query #1656

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 40 additions & 46 deletions backend/src/entities/cedar-authorization/scripts/migrate-permissions-to-cedar.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,65 +1,59 @@
import { DataSource } from 'typeorm';
import { AccessLevelEnum, PermissionTypeEnum } from '../../../enums/index.js';
import { ConnectionEntity } from '../../connection/connection.entity.js';
import { GroupEntity } from '../../group/group.entity.js';
import { PermissionEntity } from '../../permission/permission.entity.js';
import { IComplexPermission, ITablePermissionData } from '../../permission/permission.interface.js';
import { generateCedarPolicyForGroup } from '../cedar-policy-generator.js';

export async function migratePermissionsToCedar(dataSource: DataSource): Promise<void> {
const connectionRepository = dataSource.getRepository(ConnectionEntity);
const groupRepository = dataSource.getRepository(GroupEntity);
const permissionRepository = dataSource.getRepository(PermissionEntity);

const connections = await connectionRepository.find();
let migratedCount = 0;

for (const connection of connections) {
const groups = await groupRepository
.createQueryBuilder('group')
.leftJoinAndSelect('group.connection', 'connection')
.leftJoinAndSelect('group.permissions', 'permission')
.where('connection.id = :connectionId', { connectionId: connection.id })
.andWhere('(group.cedarPolicy IS NULL OR group.cedarPolicy = :empty)', { empty: '' })
.getMany();
const groups = await groupRepository
.createQueryBuilder('group')
.leftJoinAndSelect('group.connection', 'connection')
.leftJoinAndSelect('group.permissions', 'permission')
.where('group.cedarPolicy IS NULL OR group.cedarPolicy = :empty', { empty: '' })
.getMany();
Comment on lines +11 to +16
Copy link

Copilot AI Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This loads all groups lacking cedarPolicy (plus all their permissions) into memory at once via getMany(). If this runs during app bootstrap and the dataset is large, it can cause long startup times or OOM. Consider processing in pages/chunks (take/skip) or iterating by connection/group ids to bound memory.

Copilot uses AI. Check for mistakes.

for (const group of groups) {
const permissions = group.permissions || [];
for (const group of groups) {
const connection = group.connection;
if (!connection) continue;
Comment on lines +13 to +20
Copy link

Copilot AI Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

leftJoinAndSelect('group.connection', ...) plus if (!connection) continue; silently skips groups with missing connections. Consider using an innerJoinAndSelect (or adding a connection.id IS NOT NULL condition) so the query only returns migratable groups, and/or log/count skipped groups so the final message isn't misleading.

Copilot uses AI. Check for mistakes.

const connectionPermission = permissions.find((p) => p.type === PermissionTypeEnum.Connection);
const groupPermission = permissions.find((p) => p.type === PermissionTypeEnum.Group);
const tablePermissions = permissions.filter((p) => p.type === PermissionTypeEnum.Table);
const permissions = group.permissions || [];

const tableMap = new Map<string, ITablePermissionData>();
for (const tp of tablePermissions) {
const existing = tableMap.get(tp.tableName) || {
tableName: tp.tableName,
accessLevel: { visibility: false, readonly: false, add: false, delete: false, edit: false },
};
const level = tp.accessLevel as keyof ITablePermissionData['accessLevel'];
if (level in existing.accessLevel) {
existing.accessLevel[level] = true;
}
tableMap.set(tp.tableName, existing);
}
const connectionPermission = permissions.find((p) => p.type === PermissionTypeEnum.Connection);
const groupPermission = permissions.find((p) => p.type === PermissionTypeEnum.Group);
const tablePermissions = permissions.filter((p) => p.type === PermissionTypeEnum.Table);

const complexPermission: IComplexPermission = {
connection: {
connectionId: connection.id,
accessLevel: (connectionPermission?.accessLevel as AccessLevelEnum) || AccessLevelEnum.none,
},
group: {
groupId: group.id,
accessLevel: (groupPermission?.accessLevel as AccessLevelEnum) || AccessLevelEnum.none,
},
tables: Array.from(tableMap.values()),
const tableMap = new Map<string, ITablePermissionData>();
for (const tp of tablePermissions) {
const existing = tableMap.get(tp.tableName) || {
tableName: tp.tableName,
accessLevel: { visibility: false, readonly: false, add: false, delete: false, edit: false },
};
Comment on lines +29 to 33
Copy link

Copilot AI Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tp.tableName can be an empty string (PermissionEntity defaults tableName to ''), which would collapse multiple table permissions into the same map key and generate an incorrect policy. Add a guard to skip/log table permissions with missing/blank tableName (or treat them as invalid data).

Copilot uses AI. Check for mistakes.

const cedarPolicy = generateCedarPolicyForGroup(group.id, connection.id, group.isMain, complexPermission);
group.cedarPolicy = cedarPolicy;
await groupRepository.save(group);
migratedCount++;
const level = tp.accessLevel as keyof ITablePermissionData['accessLevel'];
if (level in existing.accessLevel) {
existing.accessLevel[level] = true;
}
tableMap.set(tp.tableName, existing);
}

const complexPermission: IComplexPermission = {
connection: {
connectionId: connection.id,
accessLevel: (connectionPermission?.accessLevel as AccessLevelEnum) || AccessLevelEnum.none,
},
group: {
groupId: group.id,
accessLevel: (groupPermission?.accessLevel as AccessLevelEnum) || AccessLevelEnum.none,
},
tables: Array.from(tableMap.values()),
};

const cedarPolicy = generateCedarPolicyForGroup(group.id, connection.id, group.isMain, complexPermission);
group.cedarPolicy = cedarPolicy;
await groupRepository.save(group);
migratedCount++;
}

console.log(`Migrated Cedar policies for ${migratedCount} groups (skipped groups with existing policies)`);
Expand Down
Loading