Affinity SolutionsRemote
Senior Data and Software Engineer
United States
Full-time
8 hours ago
$185,000 - $200,000 USD
H1B Sponsor Likely
Key skills
SQLPythonAWSSnowflakeData securityData governanceData clean roomsCI/CD processesJavaScriptAPIsBI toolsAI conceptsModelingAIMLGenerative AITensorFlowELTData LakeLookerBITableauDomoRedshiftDatabricksAirflowdbtHadoopNext.jsNode.jsGCPJenkinsLambdaS3GlueAthenaJWTSSOSwaggerOpenAPISDLCCI/CDCollaboration
About this role
Affinity Solutions is the leading consumer purchase insights company, providing a complete view of U.S. and U.K. consumer spending through exclusive access to transaction data. The Senior Data and Software Engineer will design and maintain robust data structures and controls for privacy features, ensuring compliance with data governance standards and implementing strong data security measures.
Responsibilities:
- Design and implement data privacy features and services to enable secure multi-party collaboration, including query constraints, data clean room construction, deployment, and monitoring at scale
- Uphold data governance standards and practices, ensuring compliance with data quality standards
- Implement and enforce data security measures to protect sensitive data, including personally identifiable information (PII) and financial data
- Leverage SQL and Python programming proficiency to extract, transform, clean, and interpret large data volumes (200BIL+ records)
- Design highly performant data structures to ensure optimal storage and retrieval of data
- Utilize advanced database technologies to enhance data storage and processing capabilities
- Participate in code reviews to maintain code quality and consistency; collaborate with the infrastructure team to plan and execute deployments
Requirements:
- Master's degree in Computer Science, Engineering, Data Science, or related field/experience
- 6 or more years of experience as a back-end data/software engineer working on various data technologies, with proficiency in SQL, Python, and Javascript (Node.js)
- 2 or more years of hands-on experience with Snowflake ecosystem, including expert knowledge of SnowPipes, Streams, Views, performance tuning, data modeling, ELT pipelines, data visualizations, and standard DWH concepts, and experience implementing complex SQL stored procedures
- Minimum 3 years' experience with various AWS cloud technologies and data lake management such as S3, Lambda, Airflow, Redshift, Athena, and Glue
- Demonstratable knowledge of data clean room technologies, such as creating secure data shares using RBAC
- Knowledge of all aspects of the SDLC as well as experience with Jenkins and setting up CI/CD processes
- Experience with data security concepts such as CCPA, GDPR, SSO, and JWT
- Proficiency in data access controls, including aggregation constraints, projection policies, row access policies, column masking, and differential privacy; knowledge of data controls including semantic models, dbt, honeydew, Iceberg tables, various catalogs (glue, rest, nessie, hadoop etc), and catalog syncing concepts
- Experience with cloud environments such as SNOWFLAKE, AWS, DATABRICKS, LIVERAMP, and GCP
- Expertise building and maintaining APIs
- Knowledge of Snowflake Native apps (v6+)
- Familiarity with BI tools such as Thoughtspot, Sigma, Domo, Looker, Quicksights, Tableau
- Detailed knowledge of AI concepts such as Generative AI chatbots, cortex analyst, agent training, various LLMs, and prompt engineering
- Experience in modeling: Snowpark ML, TensorFlow
- Swagger/OpenAPI spec, Node, and Next.js a plus