Staff Engineer Data

About GitGuardian GitGuardian is a global cybersecurity scale-up. The company is based in Paris, New-York City, Boston. Among our early investors who saw our market value proposition, are the co-founder of GitHub, Scott Chacon, along with Solomon Hykes , Docker's co-founder. American and European top-tier VC firms have also invested in GitGuardian. GitGuardian leads the way in Non-Human Identity security , offering end-to-end solutions from secrets detection in code, productivity tools and environments to strong remediation, observability and proactive prevention of leaks. Our solutions are already used by more than 600K developers worldwide! About your team and your mission You will join the Public Intelligence team, whose mission is to leverage public data to detect exposed secrets, map them to the correct company, assess their severity, and enable timely and relevant alerts for customers and prospects. The team works on ingesting public data (notably from GitHub and other sources), identifying the owning organization behind exposed secrets, analyzing the impact of these exposures, and evolving current systems toward more agentic and real-time architectures. The existing systems are mature and battle-tested. We are now at a pivotal moment: the goal is to redesign the end-to-end architecture to make it more robust, scalable, and aligned with significantly larger ambitions, including an agentic layer. Key challenges : Evolve the system from a deterministic approach to agentic systems, improving secret-to-company mapping accuracy and impact analysis. Redesign an existing multi-service architecture into a horizontally scalable and maintainable system. Move from batch processing to real-time processing, enabling secrets to be qualified within minutes of detection on GitHub. Extend the pipeline to new public data sources (Docker Hub, NPM, PyPI, etc.), beyond GitHub. Build a search-oriented data architecture capable of handling hundreds of millions of secrets. Scale the system to full dataset coverage, whereas only a subset is currently processed. In short: you will have real ownership over the architectural decisions that will define the next generation of the pipeline. Your responsibilities : Design and implement the end-to-end data architecture. Build real-time systems, from design through production deployment. Be hands-on on the most complex and critical technical challenges. Design monitoring, maintenance, and alerting systems around the data pipeline. Mentor and raise the technical bar within the team through code reviews and knowledge sharing. Structure engineering processes and facilitate collaboration across backend, ML, Data, and product teams. Contribute to the technical roadmap in close collaboration with the Engineering Manager and Product Manager. Technical environment Backend: Python + Django, Go, RabbitMQ, Redis DB: Elasticsearch (+ Kibana), PostgreSQL, ClickHouse, Snowflake Frontend: React / Typescript Deployment: Docker, Terraform, AWS About you If you think you match at least 70% of these criteria, please apply! Here's what we consider essential for success in this role: 7+ years of experience in data engineering, with a strong track record building and operating large-scale, production-grade data pipelines. Strong expertise in distributed systems and real-time architectures (streaming, event-driven systems). Strong experience with AWS, Terraform, Docker, and Kubernetes in production environments. Strong experience with ClickHouse or similar large-scale analytical databases. Hands-on engineering mindset: ability to contribute directly to production code on critical systems, not just design. Proven experience redesigning or refactoring existing production architectures. Strong ownership mindset: you proactively identify problems and drive solutions end-to-end. Experience mentoring engineers and raising the technical level of a team. Excellent communication skills and ability to work effectively with backend, ML,