Data sources. Data Sources: Twitter, Facebook, Flickr, Weibo, Instagram, TripAdvisor, .... Considerable effort required
Big Data approaches to Reef monitoring Susanne Becken, Griffith Institute for Tourism Rod Connolly, Australian Rivers Institute Bela Stantic, Institute for Integrated and Intelligent systems
Reporting on two projects § ‘Human sensors’ for monitoring Great Barrier Reef environmental changes and quality of marine waters through harnessing Big Data analysis (2.3.2) § Monitoring aesthetic value of the Great Barrier Reef by using artificial intelligence to score photos and videos (3.2.3)
Data sources Data Sources: Twitter, Facebook, Flickr, Weibo, Instagram, TripAdvisor, Text-based information
Image/video -based information
Analytical tasks • Topic detection (target) • Keyword analysis • Sentiment polarity (machine learning, lexicon, hybrid)
Analytical tasks • Identify Region of Interest or specific elements (AI) • Quantification of key elements • Score or rate attributes
Integration of data sources - Parallel representation (e.g. data overlay for eyeballing) - Statistical integration (weighting, reconciling, modelling)
Metadata: Geographic location, demographics
Analytical tasks • Geo-time stamp • Account holder (different methods) • Population of users and bias control
Monitoring typology
Major achievements to date – IT science § § § § §
Employed a range of APIs for obtaining data from diverse social media platforms and open data. Developed methods for crawling the web to collect data where API are not available. Developed methods to process data to incremental hierarchical structures to ensure fast visualisation. Improved the speed and accuracy of sentiment VADER algorithm and semi-supervised machine learning for target/aspect detection. Developed a novel Deep learning CNN structure for image aesthetics assessment using the Inception module and connecting intermediate local layers to the global layer.
What are the applications?
Griffith Institute for Tourism | World-leading tourism research
Socio-economic and visitor monitoring
Griffith Institute for Tourism | World-leading tourism research
Sentiment analysis
Griffith Institute for Tourism | World-leading tourism research
Red flags – acute or long-term
Example: “coral bleaching” + “GBR” in global tweets
Example: “Great Barrier Reef” in SinoWeibo posts
1) Tweets
3) Coral Watch
Integrated data systems
2) Eye on the Reef Sightings
Twitter Coral Watch EoR Sighting Reef Health
Latitude
4) Reef Health
Monitoring aesthetic value Eye tracking findings (example) Stimuli Beautiful1 Beautiful2 Beautiful3 Beautiful4 Beautiful5 Beautiful_mean Ugly1 Ugly2 Ugly3 Ugly4 Ugly5 Ugly_Mean
Total fixation time (Seconds) 3.10476 4.49331 3.97639 3.54937 3.53711 3.732188 2.85166 1.75065 2.33818 2.45623 2.27917 2.335178
Fixation count (number) 9 13.4 10.6 11 10.2 10.84 7.8 6.6 7 7.6 6.6 7.12
Automated identification § Using method of Deep Convolutional Neural Network (CNN) § Spatial pyramid pooling for the final feature computation § Trained algorithm using a large number of scored images (N=2000)
User-supplied images § Flickr (2016): 6,390 pictures, 1,440 with latitude and longitude
Automated image processing - monitoring Use cases: - Environmental monitoring at large scale - Creating new experiences for visitors - Alerts (e.g. shark or jellyfish)
Fish ID Common name (Australia)
Scientific name
GBRMPA Fully Annotated indicator automated
Bigeye trevally Blacktip reef shark
Caranx sexfasciatus No Carcharhinus melanopterus No
Blue tang Checkerboard wrasse Coral rabbitfish Coral trout Dark surgeon Green turtle Maori wrasse Red lionfish Steephead Parrotfish
Paracanthurus hepatus Halichoeres hortulanus Siganus corallinus Plectropomus laevis Acanthurus blochii Chelonia mydas Cheilinus undulatus Pterois volitans Chlorurus microrhinos
Yes No Yes Yes Yes No Yes No Yes
Whitetip reef shark
Triaenodon obesus
No
Y
Y Y
Y Y Y Y Y
Y
Y Y Y
GREAT BARRIER REEF
Conclusion and next steps § Proof of concept that social media data can be used for monitoring § Considerable effort required for data cleaning and extracting meaning § Connect social media data with existing systems (e.g. socio economic data; advocacy/conservation campaigns; citizen science) § Automated photo/video scoring and identification is most promising for environmental end uses
Key outputs to date § § § §
§ §
Becken, S., Stantic, B., Chen, J. Alaei, A.R. & Connolly, R. (2017). Monitoring the environment and human sentiment on the Great Barrier Reef: assessing the potential of collective sensing. Journal of Environmental Management. 203, 87-97. Alaei, A., Becken, S. & Stantic, B. (2017). Sentiment analysis in tourism: Beginning of a new research paradigm? Journal of Travel Research. In print. Chen, J., Stantic, B., Wang, S. (2017), Connecting Social Media Data with Observed Hybrid Data for Environment Monitoring. International Symposium on Intelligent and Distributed Computing, (IDC) 2017. DOI 10.1007/978-3-319-66379-1 Becken, S., Alaei, A., Chen, J., Connolly, R. & Stantic, B. (2017). The role of social media in sharing information about the Great Barrier Reef. August 2017. Griffith Institute for Tourism Research Report No 14. https://www2.griffith.edu.au/institute-tourism/publications/research-report-series Scott, Noel, Zhang, Rui & Le, Dung (2017). Eye-tracking: A review. Current Issues in Tourism. DOI: 10.1080/13683500.2017.1367367 R. Mandal, R. Connolly S. Becken, and B. Stantic,"Assessing fish abundance from underwater video using deep neural networks". International Joint Conference on Neural Networks (under review).
