Senior Data Engineer (Databricks)

Project description: A tech firm revolutionizing scientific research by integrating lab instruments, software, and data apps in biopharma, fostering accelerated discoveries. Responsibilities: * Own, prototype, and implement customer solutions. * Research and prototype data acquisition strategy for scientific lab instrumentation. * Research and prototype file parsers for instrument output files (.xlsx, .pdf, .txt, .raw, .fid, and many other vendor binaries). * Design and build data models. * Design and build Python data pipelines, unit tests, integration tests, and utility functions. * Build visualization, report, and dashboards using Spotfire, Tableau, Jupyter notebook and etc. * Work with the customer to test and make sure the solution fulfills their requirements and solves their need. * Coordinate project kickoff meetings; manage the customer relationship throughout the project, and conduct formal project closeout meetings. * Facilitate internal project post-mortems to identify areas of improvement on the next implementation. Mandatory requirements: * 8+ Sr. Engineer > 5 years in Python and SQL. * Passionate about science and building solutions to make the data more accessible to end-users. * Elasticsearch, science background, or experience with scientific instruments. * Experience with tools like Spotfire, Tableau, Jupyter notebook (any of them). * Undergraduate degree in Chemistry, Biology, Computer Science, Statistics, Public Health, etc. * Excellent communication skills, attention to detail, and the confidence to take control of project delivery. * Quickly understand a highly technical product and effectively communicate with product management and engineering. * Proactive problem-solving skills. * High-bandwidth: thrives when managing multiple simultaneous projects. * Intellectually curious unwavering drive to learn more every day. * Ability to think creatively about how to solve projects risks. * Experience with Snowflake, Lakehouse, and Databricks architecture. Good to have: * Graduate degree in Chemistry, Biology, Computer Science, Statistics, Public Health, etc. Tech stack: * Databricks * AWS * AWS REDSHIFT (hands-on experience required)* ECS * S3* Athena * RDS * ETL/ATL (Airflow) Python: * Python Pipelines Jobs * Python 3+ DB/SQL: * SQL/TSQL * Databases MySQL, MariaDB, Aurora, PostgreSQL, ±MS SQL * Key-Value Databases * Non relational DBS