annjan.in

Annjan Arora · data & AI engineer

I build data pipelines that stay up, and the apps people use on top of them. Databricks certified, published, and quick to pick up new tools.

Since August 2026, designing the software and detection pipeline for an ICMR-funded diagnostic kit at IIT Kharagpur. Before that, a year of production data engineering at Cognizant. I also build open-source tools for AI coding agents, like a DPDP Act compliance skill. Based in Kharagpur, open to relocation.

See the work Email me
pipeline log

    Work

    Aug 2026 — today

    Project Research Scientist - I ICMR-NIV, ICMR MedTech-funded research project, IIT Kharagpur

    • Design the software and algorithm layer of a LAMP-based microfluidic point-of-care kit for on-site virus detection: HPV, dengue, chikungunya, Zika and hepatitis.
    • Defined the detection pipeline: image → microchamber detection → colour extraction → classification → QC gate → result.
    • Built an offline-first React PWA that runs the full pipeline end to end on mock data, with an experimental colour classifier.
    • Labelling a ~4,000-image dataset to train a YOLOv8n microchamber detector, and preparing a 10,000+ image annotated dataset.
    • Wrote the project's technical report and algorithm roadmap.

    Jul 2025 — Aug 2026

    Programmer Analyst, Data Engineering Cognizant Technology Solutions, Chennai

    • Monitored and validated production data pipelines for a US healthcare client: 15+ downstream systems, millions of records a day, 99%+ uptime.
    • Redesigned 100+ broken transformation and service mappings that had stalled an SAP ECC → S/4HANA migration: about 60% fewer data inconsistency errors, and go-live stayed on schedule.
    • Integrated Databricks (EDF) with the Informatica workflows, making downstream processing about 35% faster.

    May — Sep 2024

    NPTEL intern, Cloud & IoT data engineering IIT Kharagpur

    • Built MQTT publish-subscribe pipelines on AWS (EC2, S3, IAM) for low-latency ingestion from ESP32 and Raspberry Pi devices.
    • Built a real-time Streamlit dashboard on the live sensor streams, cutting manual monitoring by about 70%.

    Projects

    Pick a project to see how it's wired.

    • Lakehouse platform

      Databricks, Delta Lake

      A bronze / silver / gold lakehouse that keeps every version of every customer.

      Code on GitHub

    • Orchestrated ETL platform

      Airflow, Postgres

      A four-stage DAG that extracts, validates, transforms and loads three million New York taxi trips.

      Code on GitHub

    • Real-time streaming service

      Kafka, Spark

      Sub-second aggregations of order events by product and region, while they are still arriving.

      Code on GitHub

    • DPDP compliance skill for AI agents

      Agentic AI, open source

      Teaches coding agents to check whether India's DPDP Act applies to a codebase, then fix the gaps and prove the fixes.

      Project page Code on GitHub

    • Point-of-care diagnostic app

      Computer vision, PWA

      Reads a microfluidic test kit from a photo and gives a result where there is no lab and no network.

    • Grounded study assistant

      Retrieval, LLM

      A docs chatbot for an internal learning platform that never presents undocumented answers as fact.

    • Fraud detection pipeline

      Streaming ML

      Scores transactions as they stream in, and explains every flag.

    • Integrated Visionary Command System

      Computer vision

      Hands-free computer control: the nose moves the cursor, a blink clicks, the voice does the rest.

    • Medical platform

      Django

      Patient records and healthcare workflows, built during my IIT Kharagpur internship.

      Code on GitHub

    • Smart gas leakage detection

      IoT, published

      Industrial gas monitoring with real-time alerts. Published at IEEE ICIMIA 2023.

    • Proof-of-work transaction system

      Flask

      A small blockchain you can watch mine, from web form to ledger.

      Code on GitHub

    • Threat modeling platform

      Security

      Finds the weak points in a system design before an attacker does.

      Code on GitHub

    Stack, by where it sits in the flow

    Ingest

    Kafka, MQTT, Informatica, IBM DataStage, Lakeflow Connect

    Process

    Databricks, Spark & PySpark, Structured Streaming, PyArrow validation

    Store

    Delta Lake, medallion lakehouse, star schemas, SCD Type 2, PostgreSQL, MySQL, MongoDB

    Model

    YOLOv8, MediaPipe, image dataset labelling, colour-based classification

    Serve

    React & offline-first PWAs, Django, Streamlit dashboards

    Ship

    Airflow, Docker & Compose, AWS (EC2, S3, IAM), Kubernetes, Terraform, Git, CI/CD

    Written in Python and SQL, with JavaScript and Bash where they fit. Domains: US healthcare data, SAP migration, medical diagnostics, industrial IoT.

    Annjan Arora, software and data engineer

    About

    I started at Cognizant on production ETL for a US healthcare client: Informatica and Databricks pipelines that had to stay up through an SAP S/4HANA migration. It taught me to care about the parts users never see — retries, schemas, validation, tests.

    Now I build the software for a point-of-care diagnostic kit, which added computer vision and app development to my data work: the image dataset behind its detector, a classifier that reads the chambers' colour, and an app that gives a result with no network at all. I'm looking for a long-term role building reliable data systems, and I learn new tools fast.

    Education
    B.E. Computer Science Engineering, Chandigarh University, Mohali, 2025. CGPA 8.32 / 10.
    Certified
    Databricks Certified Data Engineer Associate (Apr 2026 – Apr 2028). Also Data Ingestion with Lakeflow Connect, Databricks Spark-based Data Processing, IBM DataStage & Informatica ETL (advanced), AWS Cloud Fundamentals.
    Published
    Three works, as Annjan Arora / Annjan Mongia: a marketing case study on smart gas leakage detectors (IEEE ICIMIA 2023, Scopus indexed); a book chapter on women's work-life integration (IGI Global, 2025); and a mathematics paper on fixed point analysis in quasi-partial metric spaces (TWMS Journal of Applied and Engineering Mathematics, 2026, Scopus indexed).

    Have a problem that involves data or software?

    Write to me. I read everything.

    annjan0077@gmail.com

    GitHubLinkedIn