🎯 Top 50 ADF Interview Questions & Answers
Scenario-based, real-time Azure Data Factory Interview Q&A compiled by expert trainers at Rishab Informatica Group.
Read Online – First Few Questions
📌 How to copy data from On-premises SQL Server to Azure SQL Database?
✅ Set up a Self-hosted Integration Runtime (SHIR) to connect to the on-premises SQL Server. Create linked services for both the on-prem SQL Server and Azure SQL Database. Define source and sink datasets. Add a Copy Activity in your pipeline to transfer data.
📂 Category: File Ingestion
📌 How to handle multiple file formats like CSV, JSON, and Parquet in ADF?
✅ Create parameterized datasets for each file format. Use a Switch activity to handle logic per format type. Use mapping data flows to apply transformations for each format before loading.
📂 Category: File Ingestion
📌 What causes intermittent timeout errors in ADF pipelines and how to handle them?
✅ Increase timeout settings in activities. Use a retry policy in Copy Activity. Ensure your Integration Runtime (IR) has enough resources. Check for network or firewall interruptions.
📂 Category: Error Handling
📌 How to implement incremental data loading in ADF?
✅ Use a watermark column (e.g., last_updated_date). Create a Lookup activity to get the last value. Use this value in the source query to fetch only new/changed records. Update watermark after successful load.
📂 Category: Incremental Load
📄 Prefer the full visual version?
🧠 Continue Reading
📌 How to filter files dynamically while using Get Metadata and ForEach?
✅ Use Get Metadata to list files in a directory. Add a Filter activity inside the ForEach loop. Use expressions like @endswith(item().name, '.csv') to filter only relevant files.
📂 Category: Metadata / ForEach
📌 How to handle NULL values during data flow transformation?
✅ Use the Expression Builder in data flows. Apply the isNull() or iif() functions to handle nulls. Example: iif(isNull(columnName), 'default', columnName).
📂 Category: Data Flows
📌 What is the difference between binary copy and tabular copy in ADF?
✅ Binary copy moves files as-is, without parsing content. Use it for images, PDFs, ZIPs. Tabular copy reads and writes structured data (CSV, Parquet, etc.).
📂 Category: File Ingestion
📌 How to execute stored procedures in ADF?
✅ Use the Stored Procedure Activity. Create a linked service for the database. Configure procedure name and parameters under the activity settings.
📂 Category: Activities
📌 How to perform conditional branching in a pipeline?
✅ Use If Condition Activity. Define the condition using expression language (e.g., @equals(pipeline().parameters.env, 'prod')). Specify the activities to execute in True/False branches.
📂 Category: Control Flow
📌 What is a tumbling window trigger?
✅ Tumbling window triggers run pipelines at regular, periodic intervals. They maintain state and guarantee exactly-once processing. Useful for hourly, daily scheduled processing.
📂 Category: Triggers
📌 What is the use of parameters in ADF pipelines?
✅ Parameters allow you to pass dynamic values to pipelines, datasets, and linked services. They help make your pipelines reusable and configurable. Example: passing a filename, date, or folder path from the trigger or parent pipeline.
📂 Category: Control Flow
📌 How do you secure credentials in ADF?
✅ Use Azure Key Vault to securely store and access secrets. Create a linked service to the Key Vault and reference secrets using @Microsoft.KeyVault syntax. Avoid hardcoding credentials directly in pipeline parameters.
📂 Category: Security
📌 What is the difference between Get Metadata and Lookup activity?
✅ Get Metadata extracts metadata like file name, size, lastModified. Lookup runs a query or fetches a single row from a table. Use Get Metadata for file processing logic, Lookup for control table values.
📂 Category: Metadata / Lookup
📌 How can you perform logging in ADF pipelines?
✅ Create a custom logging pipeline or activity that logs pipeline name, run ID, status, and timestamp. Send logs to Azure SQL, Log Analytics, or blob storage. Use variables, Web activity, or Stored Procedure for log insertion.
📂 Category: Monitoring
📌 How to integrate ADF with Logic Apps or Azure Functions?
✅ Use Web Activity in ADF to call Logic Apps or Azure Function HTTP endpoints. Pass headers and JSON payload as required. Validate response using subsequent If Condition or Until activity.
📂 Category: Integration
📌 What is Data Flow Debug mode?
✅ Enables real-time debugging and preview of transformation results. Runs the data flow on a Spark cluster in debug mode for testing. Useful to verify expression logic, output data, and transformations.
📂 Category: Data Flows
📌 How to skip already processed files in ADF?
✅ Maintain a control table or log with processed filenames. Use Lookup activity + If Condition to check if the file exists in logs before processing. Alternatively, move processed files to archive folders.
📂 Category: File Ingestion
📌 How to connect ADF to REST API?
✅ Use REST linked service and REST dataset. Configure relative URL, authentication (e.g., bearer token), and pagination rules. Use Copy Activity or Web Activity depending on use case.
📂 Category: Web / API
📌 What is the use of variables in ADF?
✅ Variables allow temporary storage and manipulation of values during pipeline execution. Can be set using Set Variable activity and changed via Append Variable. Useful in loops, filters, and branching logic.
📂 Category: Control Flow
📌 How do you implement SCD Type 1 and Type 2 in ADF?
✅ Use Data Flows with Surrogate Key, Conditional Split, and Sink with upsert logic. SCD Type 1: Overwrite changed records. SCD Type 2: Add new rows with current flag, effective date columns.
📂 Category: Data Flows
📌 What is the difference between pipeline parameters and variables in ADF?
✅ Parameters are passed at the start of a pipeline and cannot change during execution. Variables can change within the pipeline and are useful for loops and conditional logic.
📂 Category: Control Flow
📌 How can you optimize performance in ADF Data Flows?
✅ Use partitioning, caching, and selective transformations. Avoid unnecessary columns, filter early, and monitor debug runs. Use data flow monitoring to find bottlenecks.
📂 Category: Performance
📌 What are integration runtimes in ADF?
✅ Integration Runtime (IR) is the compute infrastructure used by ADF. Types include Azure IR, Self-hosted IR, and SSIS IR. Azure IR is managed by Microsoft, SHIR runs on-premises.
📂 Category: Integration Runtime
📌 Can you copy data from SAP or Oracle using ADF?
✅ Yes, use native connectors or OData/ODBC connectors. You may need to install and configure IR for on-prem systems. ADF supports Oracle, SAP BW, SAP HANA, SAP ECC, and others.
📂 Category: Connectors
📌 What is the difference between trigger and manual execution in ADF?
✅ Trigger-based pipelines run on schedule or event. Manual execution runs on demand from the UI or API. Use triggers for automation, manual runs for testing.
📂 Category: Triggers
📌 What are integration runtimes in ADF?
✅ Integration Runtime (IR) is the compute infrastructure used by ADF. Types include Azure IR, Self-hosted IR, and SSIS IR. Azure IR is managed by Microsoft, SHIR runs on-premises.
📂 Category: Integration Runtime
📌 Can you copy data from SAP or Oracle using ADF?
✅ Yes, use native connectors or OData/ODBC connectors. You may need to install and configure IR for on-prem systems. ADF supports Oracle, SAP BW, SAP HANA, SAP ECC, and others.
📂 Category: Connectors
📌 What is the difference between trigger and manual execution in ADF?
✅ Trigger-based pipelines run on schedule or event. Manual execution runs on demand from the UI or API. Use triggers for automation, manual runs for testing.
📂 Category: Triggers
📌 How to call another pipeline in ADF?
✅ Use Execute Pipeline Activity. Allows nesting or chaining of pipelines for modular workflows. Pass parameters to the child pipeline during execution.
📂 Category: Control Flow
📌 What is the use of Until activity in ADF?
✅ Repeats a set of activities until a condition evaluates to true. Useful for polling, retrying failed steps, or looping with a counter.
📂 Category: Control Flow
📌 How can you version control ADF pipelines?
✅ Use Git integration in ADF UI. Connect to Azure Repos or GitHub. Allows branching, pull requests, and collaboration.
📂 Category: DevOps
📌 What happens when a pipeline fails?
✅ Pipeline execution stops unless you handle errors using Try-Catch pattern. Use If Condition, Success/Failure dependencies, and Alert mechanisms to handle failures gracefully.
📂 Category: Error Handling
📌 How to send alerts or notifications from ADF?
✅ Use Web Activity to call Logic Apps or external services. Logic App can be configured to send email, Teams message, or SMS. Include pipeline name, run ID, and error message.
📂 Category: Monitoring
📌 Can ADF support delta loads in Synapse or SQL?
✅ Yes, use watermarking, modified date columns, or CDC tables. Combine Lookup, Variables, and dynamic query logic. Apply upsert or merge logic using stored procedures or data flows.
📂 Category: Incremental Load
📌 How to pass parameters to SQL query in source dataset?
✅ Use dynamic content in the query property. Reference pipeline parameters or variables using @pipeline().parameters or @variables().
📂 Category: Source & Sink
📌 What is the difference between Append Variable and Set Variable?
✅ Set Variable replaces the existing value. Append Variable adds a new item to an array variable. Useful in loops to collect values.
📂 Category: Control Flow
📌 What is the use of Retry policy in activities?
✅ Retry policy helps recover from transient errors. You can set retry count and interval between retries. Default is 0 retries.
📂 Category: Error Handling
📌 What is the difference between Lookup and Stored Procedure activity?
✅ Lookup fetches single/multiple rows from a query. Stored Procedure executes a stored proc. Use Lookup for reading control values, SP for inserts/updates.
📂 Category: Activities
📌 Can you debug pipeline failures?
✅ Yes. Use output and error tabs in monitor section. Enable verbose logging or integrate with Log Analytics. Add debug messages using Set Variable or Web Activity.
📂 Category: Monitoring
📌 Can you read/write to blob storage from ADF?
✅ Yes. Create Azure Blob Storage linked service and dataset. Configure container, folder path, file format, and authentication. Use Copy Activity or Data Flow.
📂 Category: Storage
📌 Can ADF call REST APIs with authentication?
✅ Yes. Set authentication type in REST linked service (Anonymous, Basic, OAuth2). Use headers and bearer token if needed. Web Activity is preferred for POST methods.
📂 Category: Web / API
📌 How to schedule a pipeline every 15 minutes?
✅ Use Tumbling Window Trigger or Schedule Trigger with 15-minute frequency. Specify start/end time, recurrence interval, and concurrency control.
📂 Category: Triggers
📌 How to merge two datasets in ADF?
✅ Use Join transformation in Mapping Data Flows. Supports Inner, Left, Right, Full joins. Define join conditions and projections.
📂 Category: Data Flows
📌 How to archive files after copy?
✅ Use Copy Activity followed by Move or Delete Activity. Alternatively, use Data Flow with Source → Sink → Move/Archive logic.
📂 Category: File Ingestion
📌 What is Auto Resolve Integration Runtime?
✅ Default Azure IR that automatically provisions compute in same region as source/sink. No manual configuration needed. Best for simple data movement tasks.
📂 Category: Integration Runtime
📌 What is the use of Data Preview in Data Flows?
✅ Allows preview of transformation results while designing. Helps validate logic, schema, and filter conditions. Runs on debug Spark cluster.
📂 Category: Data Flows
📌 How to handle large datasets efficiently in ADF?
✅ Use partitioning, parallelism, and staging. Avoid wide tables and unnecessary columns. Leverage polybase or copy with staging for high volume transfers.
📂 Category: Performance
📌 What are system variables in ADF?
✅ Predefined variables like pipeline().RunId, trigger().name. Useful for logging, alerts, and dynamic paths. Access them using @expression syntax.
📂 Category: Control Flow
📌 Can ADF integrate with DevOps CI/CD?
✅ Yes. Use Git integration + Azure DevOps pipelines. Export ARM templates or use Publish to deploy. Support for release automation and approvals.
📂 Category: DevOps
📌 Can ADF process Excel files?
✅ Yes, using Azure Function or custom code with Web Activity. ADF doesn’t support Excel natively. Convert Excel to CSV using Azure Functions or Logic App.
📂 Category: Connectors