AWS Glue ã¯ãå®å šãããŒãžãåã®æœåºã倿ãããŒã (ETL) ãµãŒãã¹ã§ãã客æ§ã¯åæçšã®ããŒã¿ã®æºåãããŒããç°¡åã«è¡ããããã«ãªããŸããAWS ãããžã¡ã³ãã³ã³ãœãŒã«ãæ°åã¯ãªãã¯ããã°ãETL ãžã§ããäœæããŠå®è¡ã§ããŸããAWS Glue ã§ AWS ã«ä¿åãããŠããããŒã¿ãæå®ããã ãã§ãAWS Glue ã«ããããŒã¿ãæ€åºãããé¢é£ããã¡ã¿ããŒã¿ (ããŒãã«å®çŸ©ãã¹ããŒããªã©) ã AWS Glue ããŒã¿ã«ã¿ãã°ã«ä¿åãããŸããã«ã¿ãã°åããããšãããŒã¿ã¯ ETL çšã«ããã«æ€çŽ¢å¯èœãã¯ãšãªå¯èœãå©çšå¯èœãªç¶æ ã«ãªããŸããAWS Glue ã§ã¯ã³ãŒããçæãããããŒã¿å€æãããŒã¿ã®ããŒãåŠçãå®è¡ã§ããŸãã
AWS Glue ã§ã¯ãã«ã¹ã¿ãã€ãºããã³åå©çšå¯èœã§ããŒã¿ãã«ãª Python ã³ãŒããçæãããŸããETL ãžã§ãã®æºåãå®äºãããšãAWS Glue ã®å®å šãããŒãžãåã§ã¹ã±ãŒã«ã¢ãŠããã Spark ç°å¢ã§ãžã§ããå®è¡ããããã«ã¹ã±ãžã¥ãŒã«ã§ããŸããAWS Glue ã§ã¯ãäŸåé¢ä¿ã®è§£æ±ºããžã§ãã¢ãã¿ãªã³ã°ãã¢ã©ãŒããåããæè»ãªã¹ã±ãžã¥ãŒã©ãå©çšã§ããŸãã
AWS Glue ã¯ãµãŒããŒã¬ã¹ã®ãããã€ã³ãã©ã¹ãã©ã¯ãã£ã®è³Œå ¥ãèšå®ã管çã¯å¿ èŠãããŸããããžã§ãã®å®äºã«å¿ èŠãªç°å¢ã¯èªåçã«ããããžã§ãã³ã°ããããããã客æ§ã¯ ETL ãžã§ãã®å®è¡äžã«æ¶è²»ãããã³ã³ãã¥ãŒãã£ã³ã°ãªãœãŒã¹ã«å¯ŸããŠã®ã¿ããæ¯æãããã ããŸããAWS Glue ã䜿çšãããšãããŒã¿ã¯æ°åã§åæã®ããã«å©çšã§ããããã«ãªããŸãã
ãã®ãµã€ãã§åç»ãèŠãã«ã¯ãAdobe Flash Player ãŸãã¯ææ°ã®ãã©ãŠã¶ãå¿ èŠã§ãã
ETL ãéå§ãã
AWS Glue ã®äœ¿çšãéå§ãã
ç°¡å
AWS Glue ã§ã¯ãETL ãžã§ãã®æ§ç¯ã管çãå®è¡ãšãã£ãäœæ¥ã®ã»ãšãã©ãèªååãããŸããAWS Glue ã¯ãããŒã¿ãœãŒã¹ã®ã¯ããŒã«ãããŒã¿åœ¢åŒã®ç¹å®ãã¹ããŒãããã³å€æã®ææ¡ãå®è¡ããŸããAWS Glue ã§ã¯èªåçã«ã³ãŒããçæãããããŒã¿å€æãããŒãåŠçãå®è¡ã§ããŸãã
çµ±å
AWS Glue ã¯ãAWS ã®ããŸããŸãªãµãŒãã¹ãšçµ±åãããŠããŸããAWS Glue ã§ã¯ãAmazon AuroraãAmazon RDS MySQLãAmazon RDS PostreSQLãAmazon RedshiftãAmazon S3 ã«ä¿åãããããŒã¿ããAmazon EC2 ã§å®è¡ããŠãã Virtual Private Cloud (Amazon VPC) å ã® MySQL ããŒã¿ããŒã¹ã PostgreSQL ããŒã¿ããŒã¹ããµããŒããããŸããAWS Glue ã§ã¯ãAmazon AthenaãAmazon EMRãAmazon Redshift SpectrumãApache Hive ã¡ã¿ã¹ãã¢ãšäºææ§ã®ãããã¹ãŠã®ã¢ããªã±ãŒã·ã§ã³ãšè¿œå èšå®ãªãã«çµ±åã§ããŸãã
ãµãŒããŒã¬ã¹
AWS Glue ã¯ãµãŒããŒã¬ã¹ã§ããã€ã³ãã©ã¹ãã©ã¯ãã£ã®ããããžã§ãã³ã°ããã³ç®¡çã¯å¿ èŠãããŸãããå®å šãããŒãžãåã§ã¹ã±ãŒã«ã¢ãŠããã Spark ç°å¢ã§ã® ETL ãžã§ãã®å®è¡ã«å¿ èŠãªãªãœãŒã¹ã®ããããžã§ãã³ã°ãèšå®ãã¹ã±ãŒãªã³ã°ã¯ãAWS Glue ã§åŠçãããŸãããžã§ããå®è¡äžã«äœ¿çšãããªãœãŒã¹ã«å¯ŸããŠã®ã¿ããæ¯æãããã ããŸãã
éçºè ããããããã
AWS Glue ã§ã¯ãã«ã¹ã¿ãã€ãºããã³åå©çšå¯èœã§ããŒã¿ãã«ãªãPython ãš Spark ãšããäœ¿ãæ £ããæè¡ã䜿çšããŠãã ETL ã³ãŒããçæãããŸãããŸããGlue ETL ã³ãŒãã«ã«ã¹ã¿ã ã®ãªãŒããŒãã©ã€ã¿ãŒã倿ãã€ã³ããŒãã§ããŸããAWS Glue ã§çæãããã³ãŒãã¯ãªãŒãã³ãã¬ãŒã ã¯ãŒã¯ã«åºã¥ããŠããããã䜿çšå Žæã¯åºå®ãããŸãããã©ãã§ã䜿çšã§ããŸãã
æ¡å€§ã€ã¡ãŒãžãèŠã
ãŸããAWS ãããžã¡ã³ãã³ã³ãœãŒã«ã䜿çšããŠããŒã¿ãœãŒã¹ãç»é²ããŸããAWS Glue ã§ã¯ãJSONãCSVãParquet ãªã©ã®å€ãã®äžè¬çãªãœãŒã¹åœ¢åŒãããŒã¿ã¿ã€ãã«å¯ŸããŠããããããäœæãããåé¡åã䜿çšããŠããŒã¿ãœãŒã¹ãã¯ããŒã«ããããŒã¿ã«ã¿ãã°ãæ§ç¯ã§ããŸãã
æ¡å€§ã€ã¡ãŒãžãèŠã
次ã«ãããŒã¿ãœãŒã¹ãšããŒã¿ã¿ãŒã²ãããéžæããŸããAWS Glue ã§ã¯ãPython ã§ ETL ã³ãŒããçæããããœãŒã¹ããããŒã¿ãæœåºããã¿ãŒã²ããã¹ããŒãã«é©åããããã«ããŒã¿ã倿ããŠãã¿ãŒã²ããã«ããŒã¿ãããŒãããŸããã奜ã㪠IDE ãŸãã¯ããŒãããœã³ã³ã§ã³ã³ãœãŒã«ã䜿ã£ãŠããã®ã³ãŒããç·šéããããã°ããã¹ãã§ããŸãã
æ¡å€§ã€ã¡ãŒãžãèŠã
AWS Glue ã§ã¯ã宿ç㪠ETL ãžã§ãã®ã¹ã±ãžã¥ãŒã«ãè€æ°ã®ãžã§ãã®é£çµãAWS Lambda ãšãã£ãä»ã®ãµãŒãã¹ããã®ãªã³ããã³ãã§ã®ãžã§ãã®åŒã³åºããç°¡åã«è¡ããŸããAWS Glue ã§ã¯ãã客æ§ã®ãžã§ãéã®äŸåé¢ä¿ã管çãããèªåçã«åºç€ãšãªããªãœãŒã¹ãã¹ã±ãŒã«ããããžã§ãã倱æããå Žåã«ã¯å詊è¡ãããŸãã
詳现ã«ã€ããŠã¯ãAWS Glue ã®ç¹åŸŽããã¡ãã§ãèªã¿ã«ãªããã補åããã¥ã¡ã³ããåç §ããŠãã ããã
AWS Glue ã䜿çšããŠããŒã¿ã»ããã®ã¯ãªãŒãã³ã°ãæ£èŠåã匷åãè¡ã£ãŠãåæçšã«ã¯ãªãã¯ã¹ããªãŒã ãæºåãããããã°ããŒã¿ãåŠçããŸããAWS Glue ã§ã¯ãåæ§é åããŒã¿ã«å¯ŸããŠã¹ããŒããçæãããããŒã¿ã®å€æãå¹³åŠåã匷åã®ããã« ETL ã³ãŒããäœæãããããŒã¿ãŠã§ã¢ããŠã¹ã«å®æçã«ããŒããããŸãã
AWS Glue ããŒã¿ã«ã¿ãã°ã䜿çšããŠãããŒã¿ãç§»åããã«è€æ°ã® AWS ããŒã¿ã»ããã«ããã£ãŠããŒã¿ããã°ããæ€åºããã³æ€çŽ¢ã§ããŸããããŒã¿ãã«ã¿ãã°åããããšãAmazon AthenaãAmazon EMRãAmazon Redshift Spectrum ã䜿çšããŠããã«æ€çŽ¢ããã³ã¯ãšãªã§ããããã«ãªããŸãã
ããŒã¿ã¬ã€ã¯ã¯ãæ§é åããŒã¿ããã³åæ§é åããŒã¿ã®ä¿åãšåæã®ããã®äžè¬çãªæ¹æ³ãšãªã£ãŠããŸããAmazon S3 ããŒã¿ã¬ã€ã¯ã䜿çšããå ŽåãAWS Glue ã§ã¯ãã客æ§ã®ããŒã¿ãã¹ãŠãããŒã¿ãç§»åããã«åæçšã«ããã«å©çšã§ããããã«ãªããŸããGlue ã®ã¯ããŒã©ãŒã«ãããããŒã¿ã¬ã€ã¯ãã¹ãã£ã³ããŠãGlue ããŒã¿ã«ã¿ãã°ãšåºç€ããŒã¿ã®åæãç¶æã§ããŸãããã®åŸãAmazon Athena ãš Amazon Redshift Spectrum ã䜿çšããŠããŒã¿ã¬ã€ã¯ãçŽæ¥ã¯ãšãªã§ããŸãããŸããGlue ããŒã¿ã«ã¿ãã°ã Amazon EMR ã§å®è¡ããŠããããã°ããŒã¿ã¢ããªã±ãŒã·ã§ã³ã«å¯Ÿããå€éšã® Apache Hive ã¡ã¿ã¹ãã¢ãšããŠäœ¿çšããããšãã§ããŸãã
AWS Glue ã§ã¯ãæ°ããããŒã¿ã»ããã®å ¥æãšãã£ãã€ãã³ãã«åºã¥ã㊠ETL ãžã§ããå®è¡ã§ããŸããäŸãã°ãAWS Lambda 颿°ã䜿çšããŠãAmazon S3 ã§æ°ããããŒã¿ãå©çšå¯èœã«ãªããšããã«å®è¡ããã ETL ãžã§ããããªã¬ãŒã§ããŸãããŸããAWS Glue ããŒã¿ã«ã¿ãã°å ã®æ°ããããŒã¿ã»ããã ETL ãžã§ãã®äžéšãšããŠç»é²ããããšãã§ããŸãã
AWS Glue ã®äœ¿çšãéå§ããã®ã¯ç°¡åã§ããAWS ãããžã¡ã³ãã³ã³ãœãŒã«ã«ãµã€ã³ã€ã³ããŠã[åæ] ã«ããŽãªã® [Glue] ã«ã¢ã¯ã»ã¹ããŠãã ããã

