How to run databricks Jobs API from Azure Data Factory/Triggering Databricks Jobs API from Data factory

here is the logic to be implemented in your pipeline when using data bricks Job API triggering your pipeline from ADF.

Here is the json template attached you can use in your ADF

Download the “Run Databricks Job.json” file attached and do one of the followings:

1. If you have Git configured on your Azure Data Factory: put the file under the “pipeline” folder.

If not: copy the content of the downloaded file, create a new pipeline in your ADF with the name “Rub Databricks Job” from the portal, click on “View the JSON code representation of this resource” button from the top right corner of the workspace, paste the copied code, and save the changes.

Configuration for Executing Azure Databricks Jobs from ADF
The sections below walkthrough how to build and configure a modular ADF pipeline that can execute any Azure Databricks defined job using out-of-the-box ADF pipeline activities and managed identity authentication. You can also program the pipeline yourself using the following steps.