Enterprise cloud engineering: Netflix, Arch & Atlascloud

This is the depth behind the AI work. Agents in production are still software on machines, and most of the hard failures I've seen live below the model: storage, network, cost, access.

Three engagements: Arch Platform (2016–2021), a cloud-native VFX studio platform; a Netflix contract (2018–2020) delivered through Arch; and Atlascloud, where I've been vCTO and adviser for about ten years.

I onboarded and supported a team of Netflix artists working remotely on Arch Platform. They were all on the same show, split between Vancouver, BC and Los Angeles. Both sites had to stay in sync.

• —Nightly sync, terabytes at a time. The usual answer is Aspera, which is fast and expensive. I matched its transfer speed with parallel Python rsync scripts.
• —Artists choose what syncs. A custom dashboard let artists control which data moved, so the nightly run carried what the show needed and nothing else.
• —Cloud cost −50%. Render and workstation instances sat idle overnight. Automated nightly shutdown cut the cloud bill in half.

Netflix bought the system and took it in-house. That ended our contract, which is the outcome you want from a pilot.

• —Render-farm autoscaling. The farm grew and shrank with the queue. The studio tripled its throughput without buying new hardware.
• —Provisioning in minutes. Provisioning went from hours to minutes.
• —Multi-tenant SaaS. Tenants were provisioned with Terraform.
• —Disney security approval. I led the studio's security-compliance process through Disney's review.

Atlascloud is why I don't treat infrastructure as someone else's problem. Over about ten years as vCTO and adviser I designed the dedicated servers, the hypervisors (KVM and Proxmox), the cloud and VM layer, the network design, firewalls and VLANs, the fibre, the security and the remote support.

When an AI system is slow, expensive or flaky, I can follow it from the prompt down to the switch port.

Measure the transfer cost per show from the start. We knew Aspera's licence was gone, but the nightly data-transfer bill should have been on the same dashboard the artists used to pick what synced.

• —Cloud: AWS (earlier IBM SoftLayer), Terraform
• —Sync: Python, parallel rsync, custom artist dashboard
• —Render and workstations: autoscaled render farm, cloud workstations with scheduled shutdown
• —Metal: dedicated servers, KVM and Proxmox, VLANs and firewalls, fibre