{"id":678,"date":"2018-10-22T11:12:19","date_gmt":"2018-10-22T09:12:19","guid":{"rendered":"http:\/\/cwiok.pl\/?p=678"},"modified":"2018-10-22T11:12:19","modified_gmt":"2018-10-22T09:12:19","slug":"usuwanie-plikow-z-ftp-przy-pomocy-azure-data-factory-podejscie-drugie","status":"publish","type":"post","link":"https:\/\/cwiok.pl\/index.php\/pl\/2018\/10\/22\/usuwanie-plikow-z-ftp-przy-pomocy-azure-data-factory-podejscie-drugie\/","title":{"rendered":"Usuwanie plik\u00f3w z FTP przy pomocy Azure Data Factory. Podej\u015bcie drugie."},"content":{"rendered":"

Ze wzgl\u0119du na popularno\u015b\u0107 postu, postanowi\u0142em zrobi\u0107 drugie podej\u015bcie do usuwania plik\u00f3w z FTP przy pomocy pipeline’u Data Factory. W mojej ocenie poni\u017csze rozwi\u0105zane jest lepsze, \u0142atwiejsze do debugowania i zapewnia wi\u0119ksz\u0105 kontrol\u0119.<\/p>\n

\"\"<\/a><\/div>\n

Najpierw, przeczytaj m\u00f3j artyku\u0142 o Key Vault<\/a>. Jest to istotna wiedza, poniewa\u017c b\u0119dziesz tam przechowywa\u0142 po\u015bwiadczenia do serwera FTP. Stw\u00f3rz Key Vault i dwa sekrety – jeden dla loginu i jeden dla has\u0142a. Mo\u017cesz te\u017c przechowa\u0107 tam adres IP serwera.<\/p>\n

Nast\u0119pnie, stw\u00f3rz now\u0105 Azure Function:<\/p>\n

\"image\"<\/a><\/div>\n

 <\/p>\n

Wybierz v2:<\/p>\n

\"image\"<\/a><\/div>\n

Nast\u0119pnie wklej poni\u017cszy kod do rozwi\u0105zania:<\/p>\n

using System;\r\nusing System.IO;\r\nusing System.Threading.Tasks;\r\nusing Microsoft.AspNetCore.Mvc;\r\nusing Microsoft.Azure.WebJobs;\r\nusing Microsoft.Azure.WebJobs.Extensions.Http;\r\nusing Microsoft.AspNetCore.Http;\r\nusing Microsoft.Extensions.Logging;\r\nusing Newtonsoft.Json;\r\nusing System.Collections.Generic;\r\nusing System.Net;\r\nusing Microsoft.Azure.Services.AppAuthentication;\r\nusing Microsoft.Azure.KeyVault;\r\n\r\nnamespace DeleteFromFTP_Function\r\n{\r\n     public static class DeleteFromFTP\r\n     {\r\n         [FunctionName(\"DeleteFromFTP\")]\r\n         public static async Task<IActionResult> Run(\r\n             [HttpTrigger(AuthorizationLevel.Function, \"get\", \"post\", Route = null)] HttpRequest req,\r\n             ILogger log)\r\n         {\r\n             log.LogInformation(\"C# HTTP trigger function processed a request.\");\r\n\r\n            var azureServiceTokenProvider = new AzureServiceTokenProvider();\r\n             var kv = new KeyVaultClient(new KeyVaultClient.AuthenticationCallback(azureServiceTokenProvider.KeyVaultTokenCallback));\r\n             string Login = kv.GetSecretAsync(Environment.GetEnvironmentVariable(\"FTPLogin\")).Result.Value;\r\n             string Password = kv.GetSecretAsync(Environment.GetEnvironmentVariable(\"FTPPassword\")).Result.Value;\r\n\r\n           \r\n            \r\n             string Path = req.Query[\"Path\"];\r\n             string IP = req.Query[\"IP\"];\r\n           \r\n             string requestBody = await new StreamReader(req.Body).ReadToEndAsync();\r\n             dynamic data = JsonConvert.DeserializeObject(requestBody);\r\n             Path = Path ?? data?.Path;\r\n             IP = IP ?? data?.IP;\r\n\r\n\r\n             DeleteFTPDirectory(Path + \"\/\", IP, Login, Password);\r\n             return (ActionResult)new OkObjectResult(data);\r\n\r\n           \r\n\r\n               \r\n         }\r\n         public static List<string> DirectoryListing(string Path, string ServerAdress, string Login, string Password)\r\n         {\r\n             FtpWebRequest request = (FtpWebRequest)WebRequest.Create(\"ftp:\/\/\" + ServerAdress + Path);\r\n             request.Credentials = new NetworkCredential(Login, Password);\r\n\r\n            request.Method = WebRequestMethods.Ftp.ListDirectory;\r\n\r\n            FtpWebResponse response = (FtpWebResponse)request.GetResponse();\r\n             Stream responseStream = response.GetResponseStream();\r\n             StreamReader reader = new StreamReader(responseStream);\r\n\r\n            List<string> result = new List<string>();\r\n\r\n            while (!reader.EndOfStream)\r\n             {\r\n                 result.Add(reader.ReadLine());\r\n             }\r\n\r\n            reader.Close();\r\n             response.Close();\r\n\r\n            return result;\r\n         }\r\n         public static void DeleteFTPFile(string Path, string ServerAdress, string Login, string Password)\r\n         {\r\n             FtpWebRequest clsRequest = (System.Net.FtpWebRequest)WebRequest.Create(\"ftp:\/\/\" + ServerAdress + Path);\r\n             clsRequest.Credentials = new System.Net.NetworkCredential(Login, Password);\r\n\r\n            clsRequest.Method = WebRequestMethods.Ftp.DeleteFile;\r\n\r\n            string result = string.Empty;\r\n             FtpWebResponse response = (FtpWebResponse)clsRequest.GetResponse();\r\n             long size = response.ContentLength;\r\n             Stream datastream = response.GetResponseStream();\r\n             StreamReader sr = new StreamReader(datastream);\r\n             result = sr.ReadToEnd();\r\n             sr.Close();\r\n             datastream.Close();\r\n             response.Close();\r\n         }\r\n         public static void DeleteFTPDirectory(string Path, string ServerAdress, string Login, string Password)\r\n         {\r\n             FtpWebRequest clsRequest = (System.Net.FtpWebRequest)WebRequest.Create(\"ftp:\/\/\" + ServerAdress + Path);\r\n             clsRequest.Credentials = new System.Net.NetworkCredential(Login, Password);\r\n\r\n            List<string> filesList = DirectoryListing(Path, ServerAdress, Login, Password);\r\n\r\n            foreach (string file in filesList)\r\n             {\r\n                 DeleteFTPFile(Path + file, ServerAdress, Login, Password);\r\n             }\r\n\r\n\r\n         }\r\n     }\r\n}<\/pre>\n
 <\/p>\n
Zauwa\u017c, \u017ce login i has\u0142o s\u0105 pobierane z Key Vault:<\/p>\n
            string Login = kv.GetSecretAsync(Environment.GetEnvironmentVariable(\"FTPLogin\")).Result.Value;\r\n            string Password = kv.GetSecretAsync(Environment.GetEnvironmentVariable(\"FTPPassword\")).Result.Value;<\/pre>\n
 <\/p>\n
Adresy URL sekret\u00f3w s\u0105 w zmiennych \u015brodowiskowych, kt\u00f3re mo\u017cesz znale\u017a\u0107 w ustawieniach aplikacji:<\/p>\n
\"image\"<\/a><\/div>\n
 <\/p>\n
W “applications settings”:<\/p>\n
\"image\"<\/a><\/div>\n
 <\/p>\n
Nast\u0119pny krok to publikacja funkcji. Mo\u017cesz to zrobi\u0107 z poziomu Visual Studio klikaj\u0105c prawym na projekt:<\/p>\n
\"image\"<\/a><\/div>\n
Przy publikacji wybierz Azure Function, do kt\u00f3rej chcesz wys\u0142a\u0107 swoja funkcj\u0119 lub stw\u00f3rz now\u0105. Pami\u0119taj, \u017ceby zarejestrowa\u0107 swoj\u0105 aplikacj\u0119 w <\/strong>Azure Active Directory i doda\u0107 j\u0105 do access policies w Key Vault.<\/strong><\/p>\n
Po publikacji, przejd\u017a do Azure Portal i przetestuj swoj\u0105 funkcj\u0119. Oczekuje pliku JSON w takim formacie:<\/p>\n
{\r\n\r\n\"Path\":\"<Path to the folder you want to clear>\",\r\n\r\n\"IP\":\"<IP of your FTP>\"\r\n\r\n}<\/pre>\n
 <\/p>\n
Je\u015bli nie dzia\u0142a, mo\u017cesz to naprawi\u0107 tutaj.<\/p>\n
Kolejnym krokiem jest wywo\u0142anie zapytania POST z poziomu Data Factory. Aby to zrobi\u0107, u\u017cyjesz web activity. Najpierw, skopiuj URL funkcji. Aby go zdoby\u0107, przejd\u017a do g\u0142\u00f3wnej menu funkcji i kliknij na <\/> Get Function URL:<\/p>\n
\"image\"<\/a><\/div>\n
 <\/p>\n
Skopiuj adres URL:<\/p>\n
\"image\"<\/a><\/div>\n
 <\/p>\n
Jeste\u015b gotowy, \u017ceby przej\u015b\u0107 do Data Factory. Stw\u00f3rz web activity:<\/p>\n
\"image\"<\/a><\/div>\n
 <\/p>\n
Nast\u0119pnie dodaj adres URL i Body w odpowiednim formacie.Mo\u017cesz nawet doda\u0107 elementy dynamiczne:<\/p>\n
\"image\"<\/a><\/div>\n
I to tyle! Teraz mo\u017cesz wywo\u0142a\u0107 funkcj\u0119 i sprawdzi\u0107 jak dzia\u0142a.<\/p>\n
Micha\u0142<\/p>\n","protected":false},"excerpt":{"rendered":"
Ze wzgl\u0119du na popularno\u015b\u0107 postu, postanowi\u0142em zrobic drugie podej\u015bcie do usuwania plik\u00f3w z FTP przy pomocy pipeline’u Data Factory. W mojej ocenie poni\u017csze rozwi\u0105zane jest lepsze, \u0142atwiejsze do debugowania i zapewnia wi\u0119ksz\u0105 kontrol\u0119.<\/p>\n
\"\"<\/a><\/div>\n
Read More<\/a><\/div>","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[30],"tags":[],"class_list":["post-678","post","type-post","status-publish","format-standard","hentry","category-azure-pl"],"_links":{"self":[{"href":"https:\/\/cwiok.pl\/index.php\/wp-json\/wp\/v2\/posts\/678","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cwiok.pl\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cwiok.pl\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cwiok.pl\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/cwiok.pl\/index.php\/wp-json\/wp\/v2\/comments?post=678"}],"version-history":[{"count":0,"href":"https:\/\/cwiok.pl\/index.php\/wp-json\/wp\/v2\/posts\/678\/revisions"}],"wp:attachment":[{"href":"https:\/\/cwiok.pl\/index.php\/wp-json\/wp\/v2\/media?parent=678"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cwiok.pl\/index.php\/wp-json\/wp\/v2\/categories?post=678"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cwiok.pl\/index.php\/wp-json\/wp\/v2\/tags?post=678"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}