We are looking for a Data Flow Engineer to join a project with one of our public-sector clients in Warsaw. In this role, you will design and run complex data pipelines that move, transform and deliver data across our systems. Your main toolbox will be Apache NiFi (Cloudera DataFlow), together with Kafka, Iceberg and the wider Cloudera Data Platform. You will work closely with data engineers, architects and business teams to make sure our data flows are reliable, secure and well documented.

Requirements

The ideal candidate will be responsible for:

• Designing, building, testing and maintaining complex data flows in Cloudera DataFlow (Apache NiFi) — ingest, transform, enrich, route and deliver data.
• Building and tuning Change Data Capture (CDC) pipelines in real time or near real time, using NiFi together with Kafka, Debezium or SQL-based CDC connectors.
• Connecting external systems through REST APIs, JDBC, Kafka and other protocols.
• Managing data schemas in Avro and keeping metadata and lineage clean in Apache Atlas.
• Setting up security and governance for data flows through Apache Ranger policies.
• Monitoring pipelines, setting up alerts and fixing performance or reliability issues.
• Working with data engineers, architects and business stakeholders to gather requirements and shape the architecture of data flows.
• Writing and keeping up to date SOPs, runbooks and technical documentation.
• Taking part in upgrades and migrations of CDP, NiFi and Kafka.

To excel in this role, you'll need:

• At least 6 years of relevant experience.
• Bachelor’s or Master’s degree in Computer Science, Engineering or a related technical field.
• At least one of the following certifications: Cloudera Certified Developer for Apache NiFi or Cloudera Data Flow (CFM) related certification, or equivalent.
• Skills in designing, building and maintaining complex flows in Apache NiFi, with 2-3 years of daily hands-on work, ideally in a CDP environment, and at least one large delivered integration project where NiFi was the central tool.
• Strong Python skills for data processing, custom NiFi logic, automation and integrations.
• Solid experience with REST API integrations — endpoint calls, OAuth/JWT, rate limiting and error recovery.
• Hands-on experience building CDC pipelines to and from relational databases, using native NiFi processors, connectors and SQL Builder.
• Practical knowledge of Apache Iceberg (tables, schema evolution, partitioning) and its integration with NiFi, Spark or Flink, preferably in CDP.
• Experience with data governance in CDP — Apache Atlas for metadata, lineage and tagging, and Apache Ranger for security policies and audit on NiFi flows.
• Experience with Apache Kafka as a message broker (topics, producers and consumers, schema registry, NiFi integration) and Apache Avro for serialisation and schema evolution.
• English at B2 level (CEFR) or higher.

Please notice that this role is only available for freelancing/self-employed candidates and it's onsite work in central Warsaw. The initial duration is expected to be 12 months, with possible prolongations.

If you find our proposal interesting and you meet the above requirements for the position, please don't hesitate to apply.

All applications will be treated strictly confidential.

About Comstream:

Comstream is a dynamic and growing organization with approximately 80 employees. We specialize in connecting top IT talent with ambitious growth companies, while also delivering outstanding customer support across a wide range of industries. In addition, we proudly develop and continuously improve our proprietary SaaS platform, Jobshark, designed for tech recruitment and freelancing.

Over the past few years, we’ve achieved significant growth — and we’re just getting started. If you're looking to join an international company with a welcoming welcoming, Scandinavian-inspired culture, Comstream could be the perfect fit. We’re committed to your professional development and offer an environment where you’re empowered to thrive.

At Comstream, teamwork is the foundation of everything we do. We foster a culture where every voice is heard, and where collaboration, freedom, and empowerment aren’t just values — they shape the way we work every day. Here, you’ll have the opportunity to make a meaningful impact.

For more information, please visit: www.comstream.eu