Skip to main content
KinetiStack™

Quickstart

This guide assumes you have:

  • An Azure subscription with Owner on a resource group (Contributor works with assignRoles: false — see the runbook for the split-deploy variant).
  • Oracle reachable on TCP (Azure VM, on-prem via VPN/ER, OCI, or AWS RDS) with the required grants.
  • az CLI 2.60+ and docker (only needed if you build your own images).
  • A subnet for Container Apps (≥/27, delegated to Microsoft.App/environments).

Don’t have an Oracle source? Evaluation path

For a no-Oracle-required evaluation, scripts/deploy-eval-oracle.sh provisions a complete Oracle Free 23ai container on an Azure VM, seeds EMPLOYEES + INVOICES test schemas, applies the SYS-level grants the extractor needs, and prints a main.parameters.json snippet ready to paste:

./scripts/deploy-eval-oracle.sh oradls-eval eastus2

Total provisioning time: ~15 minutes. Cost: ~$0.10/hr (D2s_v3 VM). Tear down with az group delete -n oradls-eval --yes --no-wait when done.

1. Register Azure resource providers (one-time)

./scripts/register-rps.sh

2. Fill in the parameters file

cp infra/main.parameters.example.json infra/main.parameters.json
$EDITOR infra/main.parameters.json

Mandatory fields (everything else has sensible defaults):

FieldExample
namePrefixoradls (3–11 chars, kebab-case)
extractorImageksoradlsacrprod.azurecr.io/oracle-to-adls-extractor:latest (publisher default, anonymous-pull)
controlPlaneImageksoradlsacrprod.azurecr.io/oracle-to-adls-controlplane:latest (publisher default, anonymous-pull)
acrLoginServer“ (empty when using publisher defaults — anonymous pull, no auth needed)
acrName“ (empty unless you fork to your own ACR in this RG)
oracleHost10.0.1.10 (must be reachable from the CAE subnet — run preflight job to verify)
oracleServiceORCLPDB1
oracleUserarchive_reader (see grants)
oraclePassword******** (stored in Azure Key Vault automatically; never appears in template state)
defaultSchemasHR,FINANCE
caeSubnetResourceIdfull /subscriptions/.../subnets/<your-cae-subnet>

Optional but recommended:

FieldWhat it doesDefault
authModeauto uses our custom token provider — works on Container Apps MSI, AKS WIF, VM MSI without extra configauto
lifecycleProfilestandard adds tier transitions (Hot→Cool@30d→Archive@90d); compliance adds 7-year WORMnone
peSubnetResourceIdIf set, storage + KV get private endpoints; public network is disabled. Required for HIPAA-tier deployments.empty
containerAppsEnvironmentTier (v0.14.0+)Consumption (serverless, $0 idle, max 2 vCPU / 4 GiB) or WorkloadProfile (always-warm, supports D4/D8/D16 up to 16 vCPU / 32 GiB). Switch to WorkloadProfile for archives with 500K+ row tables or LOB-heavy schemas where Consumption’s 4 GiB ceiling causes silent OOM.Consumption
workloadProfileName (v0.14.0+)D-series size, only honored on WorkloadProfile. D4 ≈ $120/mo idle, D8 ≈ $240, D16 ≈ $480.D4
extractFetchSize (v0.14.0+)JDBC rows-per-fetch buffer on extract. Lower (2000) for wide tables that OOM on 4 GiB; raise (50000+) on Workload Profile. Overridable per extract via the UI Advanced field.10000
restoreBatchSize (v0.14.0+)JDBC rows-per-INSERT batch on restore. Lower (1000) when Consumption-tier restores OOM-kill silently on 500K+ row tables — the typical failure is the Job marked Failed with no error log because Linux OOM-killed the JVM. Overridable per restore via the UI.10000

3. Deploy

az group create -n my-archive-rg -l eastus2
./scripts/deploy.sh my-archive-rg

The script runs az deployment group create against infra/main.bicep and prints the resource IDs of everything provisioned.

4. Run your first extract

az containerapp job start -g my-archive-rg -n oradls-extractor

The job runs Spark inside Container Apps, reads your Oracle schemas, and writes Parquet to ADLS Gen2. Typical durations:

Schema sizeDuration
<10 tables, <1M rows~1–2 min
100 tables, ~50M rows~10–15 min
1,700 tables, ~1 TB~90–120 min

Monitor:

az containerapp job execution list -g my-archive-rg -n oradls-extractor -o table

5. Query the archived data WITHOUT restoring to Oracle

After extract, the job writes two SQL scripts to ADLS that expose your Parquet as queryable views:

abfss://archive@<storage>.dfs.core.windows.net/<job_id>/_catalog/
  synapse.sql    # for Synapse Serverless SQL pool
  fabric.sql     # for Fabric Lakehouse SQL endpoint

See SYNAPSE-QUERY-GUIDE.md for step-by-step usage.

6. Restore back to Oracle (optional)

Restore is its own job. Same image, different env vars:

az containerapp job update -g my-archive-rg -n oradls-restore \
  --set-env-vars JOB_ID=<extract-job-id> \
                 SOURCE_SCHEMA=HR \
                 TARGET_SCHEMA=HR_RESTORED \
                 DROP_IF_EXISTS=true

az containerapp job start -g my-archive-rg -n oradls-restore

The restore reapplies the captured DDL (with portability cleanups), bulk-loads Parquet via JDBC, and validates row counts against the manifest.

7. Tear down

From the Azure Portal (Marketplace customers)

Open the resource group in the Portal → Delete resource group.

After the deletion completes (15–30 min), two resources stay in a soft-deleted state and block redeploying into the same RG:

ResourceSoft-delete windowHow to purge
Key Vault (<prefix>-kv-…)7 daysaz keyvault purge --name <vault-name>
Log Analytics workspace (<prefix>-law)14 daysaz monitor log-analytics workspace delete -g <rg> -n <workspace> --force true --yes

If you don’t need to redeploy with the same names, you can ignore them and Azure will reclaim the names after the soft-delete window expires.

The template does not create any resources outside the RG, so deleting the RG removes everything else (UAMI, role assignments, private endpoints, private DNS zones, action group, alerts). The optional grantDeployerKvAccess / grantDeployerStorageAccess role assignments are scoped to the deleted KV/storage and disappear with them.

From the CLI (CLI customers)

./scripts/teardown.sh my-archive-rg

The script:

  • Lists every resource so you confirm before delete
  • Captures KV + LAW names before the RG delete starts
  • Submits az group delete --no-wait (deletion runs async, 15–30 min)
  • Purges the soft-deleted KV + LAW automatically

For compliance scenarios where the archived data must outlive the orchestration infra:

./scripts/teardown.sh my-archive-rg --preserve-archive

This first moves the ADLS Gen2 storage account to a quarantine RG (<rg>-preserved-archive), then deletes the original RG. The preserved storage keeps the archived Parquet files indefinitely.


Oracle grants

The extractor needs a user with these privileges. EXECUTE on SYS packages must be granted by SYS AS SYSDBA in a PDB (the local administrator can’t).

GRANT CREATE SESSION        TO archive_reader;
GRANT SELECT ANY TABLE      TO archive_reader;
GRANT FLASHBACK ANY TABLE   TO archive_reader;
GRANT SELECT ANY DICTIONARY TO archive_reader;

-- SYS package execute (as SYSDBA in the PDB)
GRANT EXECUTE ON SYS.DBMS_FLASHBACK TO archive_reader;
GRANT EXECUTE ON SYS.DBMS_METADATA  TO archive_reader;
GRANT EXECUTE ON SYS.DBMS_LOB       TO archive_reader;

If the same user is also used for restore (same connection string), add:

GRANT CREATE ANY TABLE TO archive_reader;
GRANT ALTER  ANY TABLE TO archive_reader;
GRANT DROP   ANY TABLE TO archive_reader;
GRANT INSERT ANY TABLE TO archive_reader;
GRANT CREATE ANY INDEX TO archive_reader;  -- 23ai specifically requires this

For least-privilege deployments, use two users (extract = read-only, restore = write-capable) and configure each Container Apps Job with its own oracleUser.


Running multiple archives in parallel

By default each Job runs one execution at a time (jobParallelism: 1). Raise this if you want to archive several schemas in parallel:

param jobParallelism int = 4

Trade-offs:

  • Oracle PROCESSES: each parallel execution opens its own connection pool. With the default pkConcurrent=4 per execution, 4 parallel jobs can claim up to ~32 connections to the source DB. Make sure source DB PROCESSES is ≥ jobParallelism × (pkConcurrent + rowidConcurrent + lobConcurrent + 4).
  • Container Apps cost: each execution bills independently (~$0.000024/vCPU-sec). Doubling parallelism doubles per-minute cost.
  • ADLS throttling: a single account can absorb 20,000 req/s without throttling; parallel archives rarely hit this.

Most customers leave parallelism at 1 and parallelize within a single job via PK_CONCURRENT / ROWID_CONCURRENT env vars instead.


Troubleshooting

SymptomLikely causeSee
MissingSubscriptionRegistrationRPs not registeredstep 1, or run scripts/register-rps.sh
unable to pull image using Managed identityAcrPull raceset acrName param so Bicep manages it
SkuNotAvailable for VM SKUsRegion/sub capacitytry Standard_D2s_v3 in eastus2
AuthorizationFailed on role assignmentsNeed Owner / UAAset assignRoles: false; run them separately
Extract Job fails with ORA-00904: DBMS_FLASHBACK.GET_SYSTEM_CHANGE_NUMBER: invalid identifierMissing EXECUTE on SYS.DBMS_FLASHBACKre-run grants as SYSDBA
Extract Job fails with Connection refused to 169.254.169.254hadoop-azure MSI provider in Container Apps doesn’t workset authMode: auto (default in v0.5+)

For end-to-end debugging, see E2E-RUNBOOK.md § Known gotchas.