Developing a data harvester in the Amazon cloud for the automated assimilation of Florida's healthy beaches reports into the GCOOS data portal
St. John's, NL（2014）
摘要
The Florida Department of Health funds beach water sampling for 34 of Florida's coastal counties. Mote Marine Laboratory, in conjunction with the Gulf of Mexico Coastal Ocean Observing System (GCOOS) developed an automated data harvester that used Web scraping technology to capture the reported data from each monitored county and store the data in a MySQL database. The database was queried nightly and the results were used to build XML files that were ingested by GCOOS and published through the GCOOS Data Portal. In early 2013 the Florida Department of Health outsourced their Web site to a commercial service provider. The new Healthy Beaches Web site used a mash-up of Google Maps and JavaScript code and no longer returned a standard HTML document. The conversion from traditional Web page to dynamic mash-up rendered our data harvester inoperable. Our entire data pipeline had to be completely rebuilt.
更多查看译文
关键词
java,web sites,xml,cloud computing,geophysics computing,graphical user interfaces,portals,query processing,relational databases,water quality,amazon cloud,florida department of health,florida healthy beaches reports,florida coastal counties,gcoos data portal,google maps,gulf of mexico coastal ocean observing system,healthy beaches web site outsourcing,javascript code,mote marine laboratory,mysql database,web page,web scraping technology,xml files,automated assimilation,automated data harvester development,beach water sampling,data harvester inoperability,data pipeline,data storage,database query,dynamic mash-up,service provider,standard html document,databases,html
AI 理解论文
溯源树
样例
生成溯源树，研究论文发展脉络
Chat Paper
正在生成论文摘要