Quickstart
This guide assumes you have:
- An Azure subscription with Owner on a resource group (Contributor works
with
assignRoles: false— see the runbook for the split-deploy variant). - Oracle reachable on TCP (Azure VM, on-prem via VPN/ER, OCI, or AWS RDS) with the required grants.
azCLI 2.60+ anddocker(only needed if you build your own images).- A subnet for Container Apps (≥/27, delegated to
Microsoft.App/environments).
Don’t have an Oracle source? Evaluation path
For a no-Oracle-required evaluation, scripts/deploy-eval-oracle.sh
provisions a complete Oracle Free 23ai container on an Azure VM,
seeds EMPLOYEES + INVOICES test schemas, applies the SYS-level
grants the extractor needs, and prints a main.parameters.json
snippet ready to paste:
./scripts/deploy-eval-oracle.sh oradls-eval eastus2
Total provisioning time: ~15 minutes. Cost: ~$0.10/hr (D2s_v3 VM).
Tear down with az group delete -n oradls-eval --yes --no-wait when
done.
1. Register Azure resource providers (one-time)
./scripts/register-rps.sh
2. Fill in the parameters file
cp infra/main.parameters.example.json infra/main.parameters.json
$EDITOR infra/main.parameters.json
Mandatory fields (everything else has sensible defaults):
| Field | Example |
|---|---|
namePrefix | oradls (3–11 chars, kebab-case) |
extractorImage | ksoradlsacrprod.azurecr.io/oracle-to-adls-extractor:latest (publisher default, anonymous-pull) |
controlPlaneImage | ksoradlsacrprod.azurecr.io/oracle-to-adls-controlplane:latest (publisher default, anonymous-pull) |
acrLoginServer | “ (empty when using publisher defaults — anonymous pull, no auth needed) |
acrName | “ (empty unless you fork to your own ACR in this RG) |
oracleHost | 10.0.1.10 (must be reachable from the CAE subnet — run preflight job to verify) |
oracleService | ORCLPDB1 |
oracleUser | archive_reader (see grants) |
oraclePassword | ******** (stored in Azure Key Vault automatically; never appears in template state) |
defaultSchemas | HR,FINANCE |
caeSubnetResourceId | full /subscriptions/.../subnets/<your-cae-subnet> |
Optional but recommended:
| Field | What it does | Default |
|---|---|---|
authMode | auto uses our custom token provider — works on Container Apps MSI, AKS WIF, VM MSI without extra config | auto |
lifecycleProfile | standard adds tier transitions (Hot→Cool@30d→Archive@90d); compliance adds 7-year WORM | none |
peSubnetResourceId | If set, storage + KV get private endpoints; public network is disabled. Required for HIPAA-tier deployments. | empty |
containerAppsEnvironmentTier (v0.14.0+) | Consumption (serverless, $0 idle, max 2 vCPU / 4 GiB) or WorkloadProfile (always-warm, supports D4/D8/D16 up to 16 vCPU / 32 GiB). Switch to WorkloadProfile for archives with 500K+ row tables or LOB-heavy schemas where Consumption’s 4 GiB ceiling causes silent OOM. | Consumption |
workloadProfileName (v0.14.0+) | D-series size, only honored on WorkloadProfile. D4 ≈ $120/mo idle, D8 ≈ $240, D16 ≈ $480. | D4 |
extractFetchSize (v0.14.0+) | JDBC rows-per-fetch buffer on extract. Lower (2000) for wide tables that OOM on 4 GiB; raise (50000+) on Workload Profile. Overridable per extract via the UI Advanced field. | 10000 |
restoreBatchSize (v0.14.0+) | JDBC rows-per-INSERT batch on restore. Lower (1000) when Consumption-tier restores OOM-kill silently on 500K+ row tables — the typical failure is the Job marked Failed with no error log because Linux OOM-killed the JVM. Overridable per restore via the UI. | 10000 |
3. Deploy
az group create -n my-archive-rg -l eastus2
./scripts/deploy.sh my-archive-rg
The script runs az deployment group create against infra/main.bicep
and prints the resource IDs of everything provisioned.
4. Run your first extract
az containerapp job start -g my-archive-rg -n oradls-extractor
The job runs Spark inside Container Apps, reads your Oracle schemas, and writes Parquet to ADLS Gen2. Typical durations:
| Schema size | Duration |
|---|---|
| <10 tables, <1M rows | ~1–2 min |
| 100 tables, ~50M rows | ~10–15 min |
| 1,700 tables, ~1 TB | ~90–120 min |
Monitor:
az containerapp job execution list -g my-archive-rg -n oradls-extractor -o table
5. Query the archived data WITHOUT restoring to Oracle
After extract, the job writes two SQL scripts to ADLS that expose your Parquet as queryable views:
abfss://archive@<storage>.dfs.core.windows.net/<job_id>/_catalog/
synapse.sql # for Synapse Serverless SQL pool
fabric.sql # for Fabric Lakehouse SQL endpoint
See SYNAPSE-QUERY-GUIDE.md for step-by-step usage.
6. Restore back to Oracle (optional)
Restore is its own job. Same image, different env vars:
az containerapp job update -g my-archive-rg -n oradls-restore \
--set-env-vars JOB_ID=<extract-job-id> \
SOURCE_SCHEMA=HR \
TARGET_SCHEMA=HR_RESTORED \
DROP_IF_EXISTS=true
az containerapp job start -g my-archive-rg -n oradls-restore
The restore reapplies the captured DDL (with portability cleanups), bulk-loads Parquet via JDBC, and validates row counts against the manifest.
7. Tear down
From the Azure Portal (Marketplace customers)
Open the resource group in the Portal → Delete resource group.
After the deletion completes (15–30 min), two resources stay in a soft-deleted state and block redeploying into the same RG:
| Resource | Soft-delete window | How to purge |
|---|---|---|
Key Vault (<prefix>-kv-…) | 7 days | az keyvault purge --name <vault-name> |
Log Analytics workspace (<prefix>-law) | 14 days | az monitor log-analytics workspace delete -g <rg> -n <workspace> --force true --yes |
If you don’t need to redeploy with the same names, you can ignore them and Azure will reclaim the names after the soft-delete window expires.
The template does not create any resources outside the RG, so
deleting the RG removes everything else (UAMI, role assignments,
private endpoints, private DNS zones, action group, alerts). The
optional grantDeployerKvAccess / grantDeployerStorageAccess role
assignments are scoped to the deleted KV/storage and disappear with
them.
From the CLI (CLI customers)
./scripts/teardown.sh my-archive-rg
The script:
- Lists every resource so you confirm before delete
- Captures KV + LAW names before the RG delete starts
- Submits
az group delete --no-wait(deletion runs async, 15–30 min) - Purges the soft-deleted KV + LAW automatically
For compliance scenarios where the archived data must outlive the orchestration infra:
./scripts/teardown.sh my-archive-rg --preserve-archive
This first moves the ADLS Gen2 storage account to a quarantine RG
(<rg>-preserved-archive), then deletes the original RG. The
preserved storage keeps the archived Parquet files indefinitely.
Oracle grants
The extractor needs a user with these privileges. EXECUTE on SYS packages must be granted by SYS AS SYSDBA in a PDB (the local administrator can’t).
GRANT CREATE SESSION TO archive_reader;
GRANT SELECT ANY TABLE TO archive_reader;
GRANT FLASHBACK ANY TABLE TO archive_reader;
GRANT SELECT ANY DICTIONARY TO archive_reader;
-- SYS package execute (as SYSDBA in the PDB)
GRANT EXECUTE ON SYS.DBMS_FLASHBACK TO archive_reader;
GRANT EXECUTE ON SYS.DBMS_METADATA TO archive_reader;
GRANT EXECUTE ON SYS.DBMS_LOB TO archive_reader;
If the same user is also used for restore (same connection string), add:
GRANT CREATE ANY TABLE TO archive_reader;
GRANT ALTER ANY TABLE TO archive_reader;
GRANT DROP ANY TABLE TO archive_reader;
GRANT INSERT ANY TABLE TO archive_reader;
GRANT CREATE ANY INDEX TO archive_reader; -- 23ai specifically requires this
For least-privilege deployments, use two users (extract = read-only,
restore = write-capable) and configure each Container Apps Job with its
own oracleUser.
Running multiple archives in parallel
By default each Job runs one execution at a time (jobParallelism: 1).
Raise this if you want to archive several schemas in parallel:
param jobParallelism int = 4
Trade-offs:
- Oracle PROCESSES: each parallel execution opens its own connection
pool. With the default
pkConcurrent=4per execution, 4 parallel jobs can claim up to ~32 connections to the source DB. Make sure source DBPROCESSESis ≥jobParallelism × (pkConcurrent + rowidConcurrent + lobConcurrent + 4). - Container Apps cost: each execution bills independently (~$0.000024/vCPU-sec). Doubling parallelism doubles per-minute cost.
- ADLS throttling: a single account can absorb 20,000 req/s without throttling; parallel archives rarely hit this.
Most customers leave parallelism at 1 and parallelize within a single
job via PK_CONCURRENT / ROWID_CONCURRENT env vars instead.
Troubleshooting
| Symptom | Likely cause | See |
|---|---|---|
MissingSubscriptionRegistration | RPs not registered | step 1, or run scripts/register-rps.sh |
unable to pull image using Managed identity | AcrPull race | set acrName param so Bicep manages it |
SkuNotAvailable for VM SKUs | Region/sub capacity | try Standard_D2s_v3 in eastus2 |
AuthorizationFailed on role assignments | Need Owner / UAA | set assignRoles: false; run them separately |
Extract Job fails with ORA-00904: DBMS_FLASHBACK.GET_SYSTEM_CHANGE_NUMBER: invalid identifier | Missing EXECUTE on SYS.DBMS_FLASHBACK | re-run grants as SYSDBA |
Extract Job fails with Connection refused to 169.254.169.254 | hadoop-azure MSI provider in Container Apps doesn’t work | set authMode: auto (default in v0.5+) |
For end-to-end debugging, see E2E-RUNBOOK.md § Known gotchas.