PySparkSparkUnityAIMLAnalyticsDatabricksVersion Control
About this role
Role Overview
Own and evolve the end‑to‑end SA‑CCR architecture framework, covering RC, PFE, EAD, netting sets, CSA/collateral recognition, and supervisory parameter governance
Design and implement scalable patterns using Delta Lake, Unity Catalog, and Databricks Workflows, ensuring lineage, RBAC, versioning, and long‑term maintainability
Lead Spark / PySpark computation strategy for large derivatives portfolios, optimizing scenario throughput and computational efficiency
Design secure architectures covering entitlements, PII protection, legal‑entity segregation, and regulator‑ready audit evidence using versioned Delta tables and lineage tracking
Design architectures that support:
AI‑assisted recalculation and scenario testing
Intraday exposure analytics
Controlled experimentation without compromising auditability
Embed AI components within governed frameworks, ensuring:
Unity Catalog–based lineage and RBAC
Delta Lake version control for reproducible runs
Deterministic, reproducible, and evidence‑ready AI outputs
Apply explainable ML‑assisted techniques to optimize Spark joins, aggregations, and scenario performance—not black‑box optimization
Deep hands‑on expertise with Databricks, including Delta Lake, Unity Catalog, and Workflows
Strong Spark / PySpark engineering background for high‑volume risk calculations
Experience delivering regulatory‑defensible architectures for counterparty credit risk or derivatives portfolios
Strong understanding of data governance, security, and auditability in regulated environments
Ability to clearly defend architectural and AI design choices to Model Risk, Internal Audit, and Regulators
Strong stakeholder engagement skills across Risk, Quants, Technology, Treasury/Capital, Front Office, and Security
Tech Stack
PySpark
Spark
Unity
Benefits
medical, dental, and vision coverage to employees and dependents
a 401(k) plan with a generous employer match
an employee stock purchase plan
a generous Paid Time Off policy
paid parental leave and adoption assistance
Wellness Program supports employee total well-being by providing free annual health screenings and coaching, bank at work, and on-site workshops, as well as ongoing programs recognizing major events in the lives of our employees throughout the year.