![\"\"](\"http:\/\/cwiok.pl\/wp-content\/uploads\/2018\/10\/this-is-azuredatafactory.png\")
<\/a><\/div>\nFirst make sure, you have read my article on setting up a Key Vault<\/a>. It is crucial knownledge, as you are going to store FTP credentials there. Set it up and create two secrets \u2013 one for login and one for password. If you wish, you may also store your IP address.<\/p>\n Next, create a new Azure Function:<\/p>\n <\/p>\n Make sure to pick v2:<\/p>\n Next paste the code into the solution:<\/p>\n <\/p>\n Notice, that Login and Password are taken from the Key Vault:<\/p>\n <\/p>\n Secret URLs are retrieved from Environmental variables that you can find in application settings:<\/p>\n <\/p>\n Under applications settings:<\/p>\n <\/p>\n Then, next step is to publish you function. You can do it from within Visual Studio by right-clicking on the project:<\/p>\n In there pick the Azure Function that you want to publish to, or alternatively create a new one. Remember to register you app in Azure Active Directory and add it in access policies in your Key Vault.<\/strong><\/p>\n After publishing, navigate to Azure Portal and test your function. It expects JSON in this format:<\/p>\n <\/p>\n <\/p>\n If it does not work, you can debug it now.<\/p>\n Next step is calling a POST request from Data Factory. To do it, you will use a web activity. However, beforehand you have to copy function URL. To get it navigate to main function page and click <\/> Get Function URL.<\/p>\n <\/p>\n Copy the function from the URL filed:<\/p>\n <\/p>\n You are ready to go to your ADF pipeline and create a web activity:<\/p>\n <\/p>\n Next you should specify the URL (that you have just copied) and JSON in body with Path and IP \u2013 you can even try to set up some dynamic content here:<\/p>\n And that is it! Now you can trigger the function and see if it works.<\/p>\n","protected":false},"excerpt":{"rendered":" Due to the demand I have decided to revisit the solution that deletes files from FTP from within a Data Factory pipeline. I believe this approach is a much cleaner design. It is also easier to debug and control.<\/p>\n<\/a><\/div>\n<\/a><\/div>\nusing System;\r\nusing System.IO;\r\nusing System.Threading.Tasks;\r\nusing Microsoft.AspNetCore.Mvc;\r\nusing Microsoft.Azure.WebJobs;\r\nusing Microsoft.Azure.WebJobs.Extensions.Http;\r\nusing Microsoft.AspNetCore.Http;\r\nusing Microsoft.Extensions.Logging;\r\nusing Newtonsoft.Json;\r\nusing System.Collections.Generic;\r\nusing System.Net;\r\nusing Microsoft.Azure.Services.AppAuthentication;\r\nusing Microsoft.Azure.KeyVault;\r\n\r\nnamespace DeleteFromFTP_Function\r\n{\r\n public static class DeleteFromFTP\r\n {\r\n [FunctionName(\"DeleteFromFTP\")]\r\n public static async Task<IActionResult> Run(\r\n [HttpTrigger(AuthorizationLevel.Function, \"get\", \"post\", Route = null)] HttpRequest req,\r\n ILogger log)\r\n {\r\n log.LogInformation(\"C# HTTP trigger function processed a request.\");\r\n\r\n var azureServiceTokenProvider = new AzureServiceTokenProvider();\r\n var kv = new KeyVaultClient(new KeyVaultClient.AuthenticationCallback(azureServiceTokenProvider.KeyVaultTokenCallback));\r\n string Login = kv.GetSecretAsync(Environment.GetEnvironmentVariable(\"FTPLogin\")).Result.Value;\r\n string Password = kv.GetSecretAsync(Environment.GetEnvironmentVariable(\"FTPPassword\")).Result.Value;\r\n\r\n \r\n \r\n string Path = req.Query[\"Path\"];\r\n string IP = req.Query[\"IP\"];\r\n \r\n string requestBody = await new StreamReader(req.Body).ReadToEndAsync();\r\n dynamic data = JsonConvert.DeserializeObject(requestBody);\r\n Path = Path ?? data?.Path;\r\n IP = IP ?? data?.IP;\r\n\r\n\r\n DeleteFTPDirectory(Path + \"\/\", IP, Login, Password);\r\n return (ActionResult)new OkObjectResult(data);\r\n\r\n \r\n\r\n \r\n }\r\n public static List<string> DirectoryListing(string Path, string ServerAdress, string Login, string Password)\r\n {\r\n FtpWebRequest request = (FtpWebRequest)WebRequest.Create(\"ftp:\/\/\" + ServerAdress + Path);\r\n request.Credentials = new NetworkCredential(Login, Password);\r\n\r\n request.Method = WebRequestMethods.Ftp.ListDirectory;\r\n\r\n FtpWebResponse response = (FtpWebResponse)request.GetResponse();\r\n Stream responseStream = response.GetResponseStream();\r\n StreamReader reader = new StreamReader(responseStream);\r\n\r\n List<string> result = new List<string>();\r\n\r\n while (!reader.EndOfStream)\r\n {\r\n result.Add(reader.ReadLine());\r\n }\r\n\r\n reader.Close();\r\n response.Close();\r\n\r\n return result;\r\n }\r\n public static void DeleteFTPFile(string Path, string ServerAdress, string Login, string Password)\r\n {\r\n FtpWebRequest clsRequest = (System.Net.FtpWebRequest)WebRequest.Create(\"ftp:\/\/\" + ServerAdress + Path);\r\n clsRequest.Credentials = new System.Net.NetworkCredential(Login, Password);\r\n\r\n clsRequest.Method = WebRequestMethods.Ftp.DeleteFile;\r\n\r\n string result = string.Empty;\r\n FtpWebResponse response = (FtpWebResponse)clsRequest.GetResponse();\r\n long size = response.ContentLength;\r\n Stream datastream = response.GetResponseStream();\r\n StreamReader sr = new StreamReader(datastream);\r\n result = sr.ReadToEnd();\r\n sr.Close();\r\n datastream.Close();\r\n response.Close();\r\n }\r\n public static void DeleteFTPDirectory(string Path, string ServerAdress, string Login, string Password)\r\n {\r\n FtpWebRequest clsRequest = (System.Net.FtpWebRequest)WebRequest.Create(\"ftp:\/\/\" + ServerAdress + Path);\r\n clsRequest.Credentials = new System.Net.NetworkCredential(Login, Password);\r\n\r\n List<string> filesList = DirectoryListing(Path, ServerAdress, Login, Password);\r\n\r\n foreach (string file in filesList)\r\n {\r\n DeleteFTPFile(Path + file, ServerAdress, Login, Password);\r\n }\r\n\r\n\r\n }\r\n }\r\n}<\/pre>\n string Login = kv.GetSecretAsync(Environment.GetEnvironmentVariable(\"FTPLogin\")).Result.Value;\r\n string Password = kv.GetSecretAsync(Environment.GetEnvironmentVariable(\"FTPPassword\")).Result.Value;<\/pre>\n
<\/a><\/div>\n<\/a><\/div>\n<\/a><\/div>\n{\r\n\r\n\"Path\":\"<Path to the folder you want to clear>\",\r\n\r\n\"IP\":\"<IP of your FTP>\"\r\n\r\n}<\/pre>\n<\/a><\/div>\n<\/a><\/div>\n<\/a><\/div>\n<\/a><\/div>\n