Sitecore.SCCH.Bulk

A .NET console tool for bulk extraction of content entities from Sitecore Content Hub into RAIF (Random Access Item File) transfer files compatible with SitecoreAI Content Transfer.

Lifecycle: POC · Framework: .NET Framework 4.8 · Owner: @iva_sitecore

Why This Exists

The CMP connector synchronizes Content Hub entities one-at-a-time via Azure Service Bus. This tool enables bulk, offline migration of large entity sets — producing RAIF files on disk without requiring a live Sitecore CM instance during extraction.

Quick Start

Prerequisites

• .NET Framework 4.8 SDK
• Access to a Sitecore Content Hub instance (with OAuth credentials)

Build

dotnet build ContentHub.BulkTransfer.CLI.sln

Run

# Extract entities into a RAIF file
dotnet run --project src/Sitecore.SCCH.Bulk/Sitecore.SCCH.Bulk.csproj -- extract --config config/sample-mapping.json --connection-string "ClientId=xxx;ClientSecret=xxx;UserName=xxx;Password=xxx;URI=https://your-instance.sitecorecontenthub.cloud/"

# Export CMP mappings from a Sitecore instance (GraphQL)
dotnet run --project src/Sitecore.SCCH.Bulk/Sitecore.SCCH.Bulk.csproj -- export-mappings --sitecore-url "https://your-sitecore-instance" --sitecore-token "<bearer-token>"

# Validate mapping configuration
dotnet run --project src/Sitecore.SCCH.Bulk/Sitecore.SCCH.Bulk.csproj -- validate-config --config config/sample-mapping.json

A sample mapping configuration is provided at config/sample-mapping.json.

Repository Structure

├── config/
│   └── sample-mapping.json           # Sample mapping configuration file
├── docs/
│   ├── User Guide.md                 # Usage and operation guidance
│   └── CH to SitecoreAI Content Migration - RAIF File Quick Guide.md
├── src/
│   ├── Sitecore.SCCH.Bulk/           # Main console application
│   ├── Sitecore.SCCH.Bulk.Tests/     # Unit tests (xUnit)
│   └── Sitecore.SCCH.Bulk.IntegrationTests/  # Integration tests
└── ContentHub.BulkTransfer.CLI.sln

Key Source Directories

Directory	Purpose
src/Sitecore.SCCH.Bulk/Configuration/	Connection, rate-limit, and mapping configuration models
src/Sitecore.SCCH.Bulk/Conversion/	Property value converters (reimplemented from CMP patterns)
src/Sitecore.SCCH.Bulk/Extraction/	Entity querying via Content Hub SDK (scroll + batch fetch)
src/Sitecore.SCCH.Bulk/Http/	HTTP resilience layer (throttling, retry, auth recovery)
src/Sitecore.SCCH.Bulk/Mapping/	Entity-to-item mapping, relation/taxonomy resolution
src/Sitecore.SCCH.Bulk/Output/	RAIF file construction, bucket folder generation, failure tracking

Documentation

Document	What It Covers
docs/User Guide.md	Usage guide, command options, and end-to-end workflow
docs/CH to SitecoreAI Content Migration - RAIF File Quick Guide.md	Quick RAIF migration guidance and operational notes

How It Works

Content Hub ──(SDK query)──▶ Entity Extractor ──▶ Mapping Engine ──▶ RAIF Writer ──▶ .raif file
                                                   ├── Field mappings
                                                   ├── Relation resolution
                                                   └── Taxonomy/tag resolution

1. Connect to Content Hub via Web Client SDK (OAuth)
2. Query entities by definition type using Elasticsearch scroll API
3. Map entities to Sitecore items using CMP-style field/relation/tag mappings
4. Write TransferredItem objects into RAIF binary format on disk
5. Upload the RAIF file to a Sitecore instance via Content Transfer (separate step)

Testing

# Unit tests
dotnet test src/Sitecore.SCCH.Bulk.Tests/

# Integration tests (requires live Content Hub + Sitecore instances)
dotnet test src/Sitecore.SCCH.Bulk.IntegrationTests/

Configuration

The tool reads from appsettings.json for Content Hub connection settings and HTTP resilience tuning. Entity/field/relation/tag mappings are defined in a separate JSON mapping file (see config/sample-mapping.json).

Configuration details and examples are documented in docs/User Guide.md.