Oracle Fusion to Azure or Databricks with BICC: a production checklist

What it takes to run 200+ PVOs every week without silent gaps — ordering, network load, delete detection and verifying every delivery.

Getting one PVO out of Oracle Fusion is easy. Running a few hundred of them on a schedule — into more than one environment, without anyone noticing a gap weeks later — is a different problem. This is the checklist we use.

1. Model your PVOs as groups with dependencies

Reference data (business units, ledgers, suppliers, projects) should land before the transactions that point at it. Organize PVOs into groups with explicit depends_on relationships, then fire them in dependency levels. Group-level dependencies are more reliable than per-PVO ones — real PVO-to-PVO references are full of cycles.

2. Fire in waves, not all at once

Firing every PVO at the same moment creates a burst of thousands of simultaneous uploads. On one weekly run that burst tripped a network firewall and a large share of pushes failed. Two fixes together cut peak connections from thousands to dozens:

• Waves: fire ~12 PVOs at a time in dependency order, 30–60 seconds apart, waiting for the previous wave's jobs to finish. 205 PVOs completed in about 25 minutes.
• Connection reuse: keep one HTTPS connection per destination per worker and reuse it across the create/append/flush calls of the storage API instead of opening a new connection for every request.

3. Push to every environment, archive only on full success

If the same extract goes to DEV, TEST and QA, treat the set as one delivery: only mark a run complete (and move it out of the landing area) when every file reached every destination. A partial push should return an error so monitoring fires and the event can be retried.

4. Verify every run end-to-end

After the last wave, compare three views of the run: what was planned, what BICC fired, and what was delivered. Anything fired-but-not-delivered, or still sitting in the landing bucket, is a problem. Send one report every run — if the report doesn't arrive, that's a signal too.

5. Alert on the first error

A monitoring alarm that only fires after five minutes of continuous errors will miss a partial push — that's usually a single burst. Alert on any error, and make the message say what it most likely means.

6. Detect deletes separately

Incremental extracts only see inserts and updates. BICC's Active Primary Key Extract job type returns the current set of primary keys for a PVO; compare it with what you've loaded to find deleted rows, and soft-delete them downstream.

7. Keep credentials out of the pipeline

• Store storage SAS tokens and SMTP credentials in a vault; grant read access to exactly the secret each component needs.
• Use service accounts, not personal logins, for BICC and for alert email.
• Rotate SAS tokens before they expire — and make sure the identity that signs them actually has data-plane access, or every new token fails with a permission error.

8. Report row counts every run

A per-PVO status and row-count email (with a CSV attached) after each run gives data owners confidence and catches anomalies — like a daily-rate PVO suddenly extracting 35,000 rows every run because its watermark froze.