Improving field training of fish identification with tablet technology

EXECUTIVE SUMMARY

Title: Improving field training of fish identification with tablet technology and assessing effectiveness of this method compared to traditional training

Report Date: March 4, 2020

Project Number: R/21-1-17

PIs: Dr. Juan J. Cruz-Motta, University of Puerto Rico, Mayagüez, Dr. Chelsea Harms-Tuohy, Isla Mar Research Expeditions, LLC, Evan Tuohy, Isla Mar Research Expeditions LLC

Proposed Start Date: January 2017 / Funds Received: August 2017 / End Date: January 2020

A. Objectives: 1) Incorporate new features and fish illustrations – by contracting two local illustrators, Daniel Irizarry and Cynthia Gotay, to illustrate a total of 116 Caribbean reef fish. 2) Involve student participation in testing prototypes – completed by recruiting volunteer graduate students from the Dept. of Marine Sciences at UPRM to participate by providing feedback on the use of the application during its conversion to the web-based format. 3) Convert the application to web-based – completed by contracting an app developer to convert the iOS software originally designed for Artedi into a web-based platform that can be accessed by an y type of tablet user. 4) Create an interactive website for users to track learning progress – completed by contracting a web designer to design the backend processing of the website to allow users to view and manipulate their data 5) Determine the effectiveness and efficiency of the app in teaching fish ID compared to traditional methods – completed by coordinating a dive plan led by the two graduate students supported from the project by which volunteer student divers participated in fish surveys with and without the experts to compare their survey results against themselves through time, each other and the experts. 6) Assess the quality of data collected and its potential for use by managers and other down-stream interest groups – completed by assessing the change in the quality of the fish survey per user over time to assess their relative contri bution next to the expert. 7) Develop an electronic survey template for expert users – completed by contracting web developer to create a digital fish survey template similar to the NOAA belt transect survey sheet. 8) Allow users to build their own surveys, however, this objective was removed from the project because it was determined to be out of the scope of the goal of Artedi.

B. Advancement of the Field: This technology was found to be an exceptional method of teaching fish identification to new surve yors and would have great contribution to citizen scientists. This project identified that Artedi is capable of competing with traditional methods of teaching fish ID and was observed to allow users to advance at a faster rate than compared to those surveyors only using traditional methods. Artedi was especially effective in training surveyors that had no experience identifying fish species.

C. Problems encountered: The project was awarded in March 2017, but the funds were not received until August 2017 so there was a significant initial delay in getting started on the

project. Hurricane Maria resulted in an additional setback wherein the illustrator and app developer were not able to work on the project while electricity was out in Puerto Rico. The methodology for evaluating the effectiveness of Artedi was reduced because of weather conditions and complicated diver scheduling such that only 2 sets of evaluation dives and 1 set of training dives occurred.

D. List PI’s supported: J.J. Cruz Motta, Chelsea Harms-Tuohy, Evan Tuohy.

E. List students supported: 1) Manuel Olmeda: manuel.olmeda@upr.edu, Msc in Marine Sciences. 2) Fernando Melendez: fernando.melendez1@upr.edu, Msc in Marine Sciences.

Category # of new students # of continuing students # of degrees awarded

Sea Grant

Supported PhD

Sea Grant

Supported MS/MA 2

Sea Grant

Supported BA/BS

Other

F. Theses/Dissertations: No theses or dissertations resulted from this project

G. Presentations: 1) Harms-Tuohy, C.; Tuohy, E. and Cruz-Motta, J.J. Improving Field Training of Caribbean Fish Identification with Tablet Technology. 2018 Southern Division American Fisheries Society Meeting. San Juan, Puerto Rico, March 7-11 (https://sd.fisheries.org/puerto-rico-2018-meeting/)

2) Fish Identificación Workshop (April 14, 2018)- Led by Isla Mar, 6 hour workshop designed to help the students learn how to identify fish using taxonomic characteristics. This workshop was created upon request from the students.

H. Publications & Other Products:

1) A documentary of the project was produced by Isla Mar – https://vimeo.com/378146060

2) Harms-Tuohy, C., Tuohy, E., Olmeda, M., Melendez, F., Cruz-Motta, J.J. In prep. Effectiveness of a new educational tool to train citizen scientist to do underwater visual censuses of fish in the Caribbean. Journal: TBD.

I. Matching Funds: $23,362.00

J. Extramural Funds: No extramural funds were obtained or secured for future work

K. Extramural Matching Funds: No extramural matching funds were secured for future work.

L. PI Time Breakdown:

Juan J. Cruz Motta: 0.66 month/year paid in full by matching components (UPRM)

Chelsea Harms-Tuohy: 1.5 month/year paid in full by SeaGrant

Evan Tuohy: 1.5 month/year paid in full by SeaGrant

Fernando Melendez: 0.33 month/year paid in full by SeaGrant

Manuel Olmeda: 0.33 month/year paid in full by SeaGrant

M. Research Impacts/Accomplishments:

RECAP – All objectives were completed with the exception of one that was removed. Artedi is now available online at www.artediapp.com and the survey template version can be accessed at pro.artediapp.com

RELEVANCE – Marine ecology is a field of experimental research using field methods that often require extensive training and proficiency to master. Such skills include underwater visual census (UVC) to characterize fish communities in marine systems, with a simultaneous application of reef fish identification (fish ID). In general, these two skills (UVC and fish ID) are taught independently, usually with fish identification driving the mastery of both techniques. Novice surveyors will often spend the majority of their survey scribbling details of an unknown fish, rather than focusing on terms to associate with the common name. Ultimately, this compromises the quantity and quality of data obtained during a survey, as more time is spent writing and quickly recording species rather than observing fish encountered during the transect. Artedi provides a solution to training UVC and fish ID.

RESPONSE – Artedi is an app that teaches UVC and fish ID simultaneously, while underwater using a tablet device in underwater housing. A website was created that allows users to upload their surveys to a cloud system where they can access them away from the tablets and without having to email them after field surveys. The website also allows users to view their data, manipulate it with graphs, and view a map of their surveys.

RESULTS – Artedi was very useful on a case-by-case basis, especially in those cases where the surveyor had no prior knowledge of fish ID. Artedi allowed those students to advance in their skills at a faster rate than the same type of surveyor using traditional methods.

FINAL REPORT

Introduction & Rationale

Marine ecology is a field of experimental research using field methods that often require extensive training and proficiency to master. Such skills include underwater visual census (UVC) to characterize fish communities in marine systems, with a simultaneous application of reef fish identification (fish ID). In general, these two skills (UVC and fish ID) are taught independently, usually with fish identification driving the mastery of both techniques. Researchers have considered the proficiency level of fish surveyors when interpreting field data (Edgar et al. 2004, Dickens et al. 2011, Bernard et al. 2013), however this consideration is not likely the case for the majority of field studies. Project leaders and natural resource managers can be limited in availability of trained personnel, opting to use a surveyor with minimal experience in fish ID versus aborting a project completely. As part of a graduate program, many students are exposed to these skills and begin to develop them over the course of their degree. However, as with any new technical skill, there is a learning curve and many students find themselves overwhelmed with the first exposure to fish identification. Novice surveyors will often spend the majority of their survey scribbling details of an unknown fish, rather than focusing on terms to associate with the common name (Figure 1). Ultimately, this compromises the quantity and quality of data obtained during a survey.

With the growing interest in citizen science and its potential use for management (Cooper et al. 2007, Dickinson et al. 2010), the ability to quickl y train non-scientists in a skill that typically requires years of practice, will be a significant improvement offered through tablet applications. The recent trend towards large scale monitoring efforts involving many professionals (including citizen scientists) with different levels of expertise, ranging over large geographical scales (e.g. Reef Check) realizes the necessity to reduce (or at least the quantification of) variation among observers. A study addressing the effect of observer experience on UVC found that experience level was correlated to estimations of fish abundance and richness (Williams et al. 2006). Specifically, Williams et al. (2006) identified that inexperienced observers counted 30% fewer fishes in certain taxa when compared with experienced observers. This study suggested that conformity among observers can be improved through better training. Another study stressed the importance of monitoring the progress of training and identifying any residual bias that may still exist between obse rvers prior to evaluating data supplied by groups of observers (Thompson and Mapstone 1997). Many citizen science projects do subject observers to a training program (i.e. REEF, Reef Check) involving presentations and workshops, with practice dives to conduct surveys However, with the

introduction of a more modern teaching approach that incorporates independent learning (Dabbagh and Kitsantas 2012), new observers now have the opportunity to test their abilities using online quizzes (e.g., www.fishid.com), or watching webinars. While these tools are useful and have helped considerably in the training process, they do not provide the observer with the opportunity to test these skills while underwater.

Apart from citizen science, natural resource managers and field biologists will also benefit from an improved method of training in fish identification. In this sense, management and funding agencies have been urging for an improved and more efficient training method for fish surveys. For example, Randy Clark, previously of the NOAA Biogeography branch stated “I think an ‘app’ would be a great training tool for newbies as well as refreshers for the pros”. When referencing the app developed prior to this proposed project, Director of Puerto Rico Sea Grant, Ruperto Chapparo commented “This project can really make a difference in fisheries management in the Caribbean by facilitating fish surveys and the training of students and volunteers”.

Prior to this project, an application existed that allowed users to experience independent instruction in fish identification while underwater. This app – Artedi – was developed by Isla Mar Research Expeditions in 2015 with the ultimate goal to train scientists, marine resource managers and citizen scientists to collect data faster, more accurately and more efficiently. Artedi bridges the gap between activities in the classroom and those in the field, providing users with a functional fish identification and reference tool for use underwater that simultaneously teaches UVC in a field setting. Additionally, Artedi improves the data collection process with an electronic format (comma-separated values, .csv) for recording and storing survey data. This makes data processing more efficient in that data becomes readily available for upload to Microsoft Excel for quantitative analysis, thus saving hours of traditional transcription of hand written data to electronic format a feature from which even the seasoned surveyor will benefit. Artedi was tested and exposed to various user groups within the first year of implementation (Figure 2). The feedback received was overwhelmingly positive, indicating both the need and desire for this technology to be incorporated into training programs. Furthermore, feedback from trained and untrained fish surveyors supported that Artedi could reduce the learning curve and training time required to learn fish ID. At the time of development, no underwater housing was available for Android tablets, thus the app was first designed for Apple systems (iPad) where the iDive underwater tablet housing (iDive Inc.) was available. Although useful, Apple tablet setups are costly (i.e. $900 for an iPad, $1000 for the iDive) and generally not within the budgets of students and citizen scientists. However, Android tablets are much more affordable (e.g.,

Samsung Galaxy Tab $200) and specific models can fit within the iDive housing. Ideally, a separate housing should be designed for the Android tablet that allows for the use of an underwater stylus that would more adequately mimic the traditional diver survey tools of waterproof paper and pencil. This transition from iOS to a new, widely available platform, like web-based applications, would make Artedi more accessible and affordable to all interest groups. Foremost, Artedi aims to facilitate instruction of fish identification and data collection in a more rapid and efficient manner. Artedi provides a template for streamlined and standardized training for all users and institutions. With the current transition to a paperless society, an electronic underwater teaching and data collection tool will make data collection processes more appealing to citizen science, educational and management institutions.

Methodology

App Development

The further development of Artedi involved extensive input, design and implementation from the developer, Matthew Katona of Better Byte Computer Co, with prototype testing orchestrated by Isla Mar. The move to a web-based platform required the use of Windows PC and Mac for testing across these platforms. Time required to develop each new app feature ranged from 1-7 hours depending on the difficulty of implementation. Programming conflicts (“bugs”) that occurred during the build process and post-build testing required between 5 minutes to 5 hours to adjust. Creative assets such as the logo, the icons, and other various branding elements required 2-8 hours to produce. The original app had to be re-created for the web (converted out of the iOS programming language) which involved designing elements similar to the native iOS user interface but creating them with web coding language. Most original functionality could be retained in this process. The app is now self-hosted on the Artedi website and can be downloaded to a user’s tablet when desired. When downloaded, the app will cache the fish illus tration images onto the tablet’s memory storage to allow the user to access the majority of the functionality without cellular or wifi access. Although the application itself was useable within Year 1 of the project, time required to finalize all user flow and better define functionality required the entire duration of the project to accomplish. Maintenance of the app will be conducted by Matthew Katona and paid for by Isla Mar for the foreseeable future.

Illustration

The illustration of the Caribbean reef fish was conducted by Daniel Irizarry and Cynthia Gotay. Daniel was an original illustrator of the app. Time to illustrate one fish required 5-9 hours and involved initial sketching before transferring to the computer to digitally render the final illustration. Isla Mar worked directly with the illustrator to identify key defining features of the fish that should be highlighted for ease of visual identification. This process included locating 34 images that capture these features for each fish, and then approving/adjusting the illustration as necessary. Most illustrations required 2-3 revisions to achieve the final, detailed and anatomically correct product. The illustration was completed in the first year of the project and

required approximately 696 hours. Finally, Isla Mar was responsible for supplying all reference information for the fish to be included in the Reference mode of Artedi. This required approximately 20 minutes per fish, for 116 fish currently on the app (~ 77 hrs total).

Website

The original website was a static website that pointed the user to the original Artedi iOS application hosted on the Apple AppStore. This project expanded that website by constructing an interactive user portal that allows surveyors access to their surveys that they have uploaded from the tablet, with some basic visual and data manipulation tools (graphs) and a survey map that shows the location of your surveys (if GPS coordinates are available). Website designer Matthew Katona (Better Byte Computer Company) created an interface that allows users to login into a personal account (via the application itself, and the backend website interface) to upload their surveys to the website. The option to download and save as a .csv file still exists, for users who wish to view their data in Excel. The website was completed in Year 2.

Experiment & Field Design

The application was tested for functionality by graduate students of the UPRM-DMS. The students did a series on underwater visual censuses, where they estimated the structure and composition of fish assemblages (i.e. surveys). In addition, an experiment was set up to evaluate the capabilities of Artedi as an educational tool; and to test whether the use of Artedi accelerated the learning process when compared with traditional methods. The experiment considered the use of the app and expertise of the user as predictor variables; and training time to reach similar capabilities as a trained surveyor, as the response variable An initial workshop was held to identify eight volunteer students and establish experimental groups of fish surveyors. A series of evaluation dives were conducted as follows: First, volunteers were randomly assigned to two teams (n=4 per team), led by a supervising expert surveyor (Harms-Tuohy or Tuohy). Surveys were conducted at random reef locations in the La Parguera Natural Reserve based on weather conditions at the time of the evaluation dive. The diversity and abundance of fish in these regions is well documented (Pittman et al. 2010; Nemeth 2013; Harms-Tuohy et al. 2016) which allows for verification of what the student surveyor and the expert are observing. Six fixed transects per site were surveyed, where transects were at least six meters apart and within 8-15 m depth. The design involved the use of three divers – two inexperienced, untrained divers (UT) and one expert (E), similar to Thompson & Mapstone (1997). The two UT surveyors proceeded shoulder-to-shoulder

along a transect line, while the third diver (E) was slightly above the two-person team (Figure 3) All divers proceeded together down the transect (25x4m) to complete 10 minutes of surveying. One of the two UT surveyors used Artedi, while the other UT diver and the E surveyor used only a traditional fish survey sheet (similar to NOAA NCRMP 2014 protocol). Each E was responsible for four UT divers, resulting in two teams (A & B), and surveys proceeded as described above, but according to the experimental design as follows: Transect 1A: UT1 and UT2 + E1. UT1 used Artedi and UT2 did not. E1 supervised and performed an independent fish survey above/behind the two UT divers. Separately, UT3 (using Artedi) and UT4 (not using Artedi) conducted Transect 1B by themselves at a different depth but within visual contact with the other team. In this scenario, UT1 and UT3 always use Artedi while UT2 and UT4 never used Artedi. Following these surveys, the team then switched transects such that: UT1 and UT4 did Transect 2A by themselves while UT2 and UT3 conducted Transect 2B supervised by E1. Teams switched a final time after concluding the second set of surveys where: UT1 and UT3 (with Artedi) recorded Transect 3A by themselves. UT2 and UT4 (both without Artedi) did transect 3B supervised by E1. The same procedure described above was conducted by Team 2, supervised by the second expert, conducted on Transects 4-6 (where UT5 and UT7 always used Artedi while UT6 and UT8 did not). The above protocol was repeated for two additional dives (n=3 dives/day). This design allowed for the testing of the following comparisons: (1) UT diver using Artedi compared to an UT conducting a traditional fish census, (2) UT using Artedi compared to an E surveyor, (3) UT without Artedi compared to E, (4) comparisons among UT divers using Artedi and finally (5) comparisons of UT divers without Artedi. The entire evaluation dive procedure was then repeated one final time at the conclusion of a series of training dives where UT1 through UT8 performed two training dives On these dives, the surveyor was accompanied by the graduate student assistant and completed a total of 2 to 3 transects per dive, 5 to 6per training day, resulting in 40 to 48 overall per person. At the conclusion of the training dives, the final set of evaluation dives proceeded as described above and supervised by Harms-Tuohy and Tuohy (n=2 dive days).

Table 1. Sampling design for survey dives used to assess the use of Artedi as an educational tool and used to compare untrained divers (UT) and expert divers (E). Divers UT1, UT3, UT5 and UT7 always used Artedi, while divers UT2, UT4, UT6, UT8, E1 and E2 never used Artedi. Transects were 25 x 4 m for a duration of 10 minutes. This protocol was executed three times (n=3 dives/day) for each team.

Transect Team 1 Divers

1A UT1, UT3, E1

1B UT2, UT4

2A UT1, UT4

2B UT2, UT3, E1

3A UT1, UT3

3B UT 2, UT4, E1

Transect Team 2 Divers

1A UT5, UT7, E2

1B UT6, UT8

2A UT5, UT8

2B UT6, UT7, E2

3A UT5, UT7

3B UT 6, UT8, E2

Data Analysis

Effect of training (independent of the training method) was expected to be reflected in the following aspects: (1) Increased observer specific precision along sequential observations, (2) Increased similarit y to an expert (trained diver), (3) Increased agreement among trainee observers, and (4) Decreased observational errors (residuals). These aspects were measured on univariate (e.g. total counts, total number of taxa, etc) as well as on multivariate estimators (e.g. similarity indices, dispersion indexes, etc).

Based on the above, the following hypotheses were tested: (1) At the end of the training, similarity between trained and UT divers will be higher for those UT using Artedi, (2) At the end of training, variability among Artedi users will be lower than among those who are not using the app, (3) All indicators described above will improve at a faster rate on users with Artedi than on those not using it and finally (4) Components of variation associated with each specific UT observer (along evaluation and training dives) will decrease at a higher rate in the group using Artedi compared to those not using Artedi.

To test those hypotheses, the following analyses were conducted on data obtained from the complex/mixed design described above: (1) Multifactorial analyses of variances based on permutations for univariate estimators (i.e. total abundances and species richness) and multivariate data (i.e. assemblage structure and composition). In this particular case, interactions between evaluation time and type of training (see design below) were of special interest. For these analyses the following design was used:

a. Evaluation Period: Initial, middle and final; fixed factor

b. Time: Two to three evaluation dive days; random factor nested within Period

c. Type of training: Artedi, without Artedi and Experts

d. Observer: 4 UT observers using Artedi, 4 UT without Artedi and 2E observers; random factor nested within Type of training

e. Replicates: 3 transects per observer, per day

Since E only participated during the initial and final periods, two variations of the analyses described above were done: 1) Only two levels of the factor period were considered (Initial and Final) but all three levels of Type were considered (Artedi, No -Artedi and Expert); 2) All three levels of Period were considered (Initial. Middle and Final), but only two levels of Type were included (Artedi and Non-Artedi). Before analyses, the data was square root transformed, to down-weight the abundant species; and standardized to account for differences in transects lengths and duration among UT.

In addition to the analyses described above performed on fish assemblages, volunteers were tested three times using theoretical quizzes. Quizzes consisted of a series of fish images for the more common species observed in La Parguera and participants were required to provide as close to exact common or species name as possible. These quizzes were given four times: 1) before the first evaluation, 2) between training dives, 3) before the second evaluation, and 4) at the conclusion of the project. Final marks per volunteer were tracked through the duration of the experiment.

Results and Findings

Web and Application

Artedi as a web-based application is now available for free at www.artediapp.com and the expert survey is available at pro.artediapp.com. The user interface for interacting and manipulating data is available through the MyArtedi portal linked from the homepage and the app itself, but requiring user credentials. The illustrators produced 116 new Caribbean reef fish illustrations to add to the ~65 illustrations that were part of the original Artedi. Additionally, reference information was added to the Reference Mode of Artedi for all newly illustrated fish.

Experiment

Multivariate analyses (PERMANOVA) of the fish assemblages data collected by UT and E, showed that despite a significant variance at the lowest spatial (i.e diver) and temporal scale (i.e time), the interaction between Period and Diver was significant (Table 2). These results were the same, independently of including experts or not in the analysis (Table 2), which indicates that temporal patterns of variations among Periods were not the same among individual divers regardless of the type of diver (Artedi, Non-Artedi or Expert). This result, also indicates, that no general conclusion can be drawn about Artedi vs Non-Artedi users. Despite these variations among individual divers, multivariate ordinations (PCO), showed that multivariate dispersions

across periods changed importantly (Figure 4), indicating not only that differences among UT divers decreased with time, but also that UT surveys were more similar to those of E during the final period than during the initial period (Figure 4). This clearly showed that UT were all trained to a satisfactory level, however, this training process did not differ between Artedi and NonArtedi users (Figure 4). In fact, and even though it was not statistically significant, during the final period, Non-Artedi UT (red triangles) appeared to be more similar to Experts (red circles) than Artedi UT (red squares) to E (Figure 4).

If temporal trajectories of the three types of divers (Arted, Non Artedi and Expert) are displayed, no clear pattern of temporal variation can be observed (Figure 5), however, it can be noted that differences among type of divers at the beginning (time 1, Figure 5) are much bigger than differences at the end of the experiment (time 13, Figure 5), indicating that all UT (in average) were effectively trained. It can also be noted, that length of E trajectory is half of that of UT (Artedi and Non-Artedi), which was expected as the E trajectory was assumed to be an accurate estimation of natural variations in fish assemblages (i.e. no effect of train ing). In this sense, it can be inferred that longer trajectories in UT (Artedi and Non- Artedi) were due to the effect of training (Figure 5). However, and contrary to our expectations, Artedi and Non-Artedi trajectories were not different (Figure 5), indicating that in average, the use of Artedi did not have an effect on the speed of learning.

Comparisons among individual UT divers and Experts showed that the learning process depended on prior knowledge and amount of study dedicated during the experiment (Figure 5). It can be noted that temporal patterns are more similar when comparing between UT with prior knowledge (or not) than when comparing between Artedi and Non-Artedi users (Figure 5).

Temporal trajectories of UT divers with no prior knowledge (regardless of using Artedi or not) were considerably longer than the trajectories of experts (Figure 5a and c) On the contrary, temporal trajectories of UT divers with prior knowledge were about the same length as those of experts (Figure 5 b and d). These results indicate that prior knowledge has a far more important effect on the learning curve than the tool that was used to learn. Now if we compare only among UT with no prior knowledge (only two cases), we can note that temporal trajectories of those using Artedi were shorter (i.e. faster learning) than those not using Artedi (Figure 7).

Table 2. Permutational multivariate analyses of variance based on Bray-Curtis dissimilarities for squareroot transformed abundances of fish species, on the basis of a mixed multifactorial model (Period: fixed, Type: fixed, Dive: random and nested in period, and Diver: random nested in Type). The probabilities associated at each pseudo-F value were obtained with 9999 permutations of residuals under a re duced model. Two results are shown: top table includes expert divers but only two periods (initial and final); lower table includes only UT divers but all three periods

Figure 4. Multivariate ordinations (PCO) of centroids per Time and Diver. Centroids are discriminated by Periods (Blue = Initial, Pink = Middle and Red = Final) and Type (Circle = Expert, Square = Artedi and Triangle = Non-Artedi). Percentages of variation are indicated in each axis.

Figure 5. Multivariate ordination of centroids of Type of diver and Time, discriminated by Type (Triangle = Non-Artedi, Square = Artedi and Circle = Expert). Numbers represent Time of sampling and lines represent temporal trajectories per Type of Diver. Percentages of variation are shown in each axis.

No Prior Knowledge

Prior Knowledge

Figure 6. Multivariate ordinations of centroids per diver and time discriminated by Type of diver (Expert = circles, Non-Artedi = triangles, Artedi = squares). Numbers indicate Periods (1 = initial, 2 = middle and 3 = final). Lines represent temporal trajectories. Percentages of variation are indicated on the axes. Left column are comparison where UT divers had no prior knowledge, whereas those on the right were comparisons with UT divers that had some prior knowledge.

Figure 7. Multivariate ordinations (PCO) of centroids per diver (only UT with no prior knowledge shown) and time discriminated by Type of diver (Non-Artedi = triangles, Artedi = squares). Numbers indicate Periods (1 = initial, 2 = middle and 3 = final). Lines represent temporal trajectories. Percentages of variation are indicated on the axes. Two different cases are presented.

Volunteer Evaluation (Quizzes)

According to the four evaluation periods, Artedi divers scored slightly higher scores in every evaluation except the training period (Q2) (Figure 8). Individual evaluations show highest score improvements in Artedi divers: Francisco Gonzalez (blue) and Khrystall Ramos (green) and Non-Artedi divers: Nicolle Lebron (purple) and Alejandro Gonzalez (yellow) (Figure 9). The highest score difference for all the individuals was observed between Q1 and Q2. Evaluation scores from Q2 to Q3 increased for all the individuals except for Jean Domenech (Gray). The lowest score differences for all the individuals are shown from Q3 to Q4 remaining constant in general (Figure 9).

Artedi vs. Non-Artedi Average Quiz Scores

Artedi Average Quiz Scores Non-Artedi Average Quiz Scores

Average Quiz Scores Quiz periods

Q1: Pre-Training Q2: Training Q3: Post-Training Q4: Final

8: Artedi and Non-Artedi Average evaluation (quiz) scores through training periods

Outreach

QUIZ SCORES

Volunteer Quiz Scores trough time

Q1: Pre-Training Q2: Training Q3: Post-Training Q4: Final

QUIZ PERIODS

Alejandro Gonzalez Alex Veglia

Fernando M Melendez

Francisco Gonzalez Jean Domenech Khrystall K. Ramos

Nicole E. Lebron Wanda M. Ortiz

Figure 9: Volunteers evaluations through training periods.

A video documentary was produced by Isla Mar using footage obtained from the duration of the project. This documentary is hosted on Isla Mar’s Vimeo channel and was premiered at the final workshop during the conclusion of the project. This final workshop provided an overview of the project and a crash course in fish identification provided to new student surveyors at the UPRMDMS.

Objectives Accomplished

The first year of the project involved improving the app Artedi by adding new features and converting the app to a web-based program to be used by all types of tablet users. There were four objectives for this portion of the project and these were addressed specifically through a contract with the CO-PIs Isla Mar who handled hiring the contractors for the illustration, app development and web development. Objective 1: Incorporate new features and fish illustrations –completed by contracting two local illustrators, Daniel Irizarry and Cynthia Gotay, to illustrate a total of 116 Caribbean reef fish. Isla Mar worked with each illustrator to provide reference images and direct illustration using their expertise as fish biologists. Objective 2: Involve student participation in testing prototypes – completed by recruiting volunteer graduate students from the

Dept. of Marine Sciences at UPRM to participate by providing feedback on the use of the application during its conversion to the web-based format. Isla Mar participated by attending the dives with these students and recording feedback thereafter to use to help direct development of intuitive features and functions of the web-based Artedi. Objective 3: Convert the application to web-based – completed by contracting an app developer, Matthew Katona of Better Byte Computer Co. to convert the iOS software originally designed for Artedi into a web-based platform that can be accessed by an y type of tablet user. Isla Mar worked directly with Matthew to design the layout, graphic elements, user flow and functionality of the app. The final version of the Artedi can now be found at www.artediapp.com by following the directions to “download” or cache the app on your tablet. The app can also be used on the computer for a virtual classroom scenario. Objective 4: Create an interactive website for users to track learning progress –completed by contracting a web designer, Matthew Katona, to design the backend processing of the website to allow users to view and manipulate their data. Isla Mar worked directly with Matthew to provide insight to the functionality desired for data viewing and manipulation that would be useful and visual to new surveyors. This portion of the app is called MyArtedi and can be accessed from the website homepage or through the app itself. User credentials also make it possible for users of the same tablet to login and record their own surveys and then upload them to their account on the website for viewing.

The second year of the project involved assessing the use of this technology in improving the learning process of fish ID. This part of the project was led by Dr. JJ Cruz-Motta and the two graduate students supported by the project. It involved dive planning and coordinating with volunteer student divers, collecting and entering data, analyzing results and assisting int the final presentation of the project. Objective 1: Determine the effectiveness and efficiency of the app in teaching fish ID compared to traditional methods – completed by coordinating a dive plan led by the two graduate students supported from the project, Manual Olmeda and Fernando Melendez, by which volunteer student divers participated in fish surveys with and without the experts ( Isla Mar) to compare their survey results against themselves through time, each other and the experts. Details described in Methods. Objective 2: Assess the quality of data collected and its potential for use by managers and other down-stream interest groups – completed by assessing the change in the quality of the fish survey per user over time to assess their relative contribution when compared to the expert. Analysis completed by Dr. Cruz-Motta, Manuel and Fernando. Objective 3: Develop an electronic survey template for expert users – completed by contracting web developer Matthew Katona to create a digital fish survey template and can be accessed at pro.artediapp.com . This survey was designed with Isla Mar input to direct its functionality for an expert user that does not need the assistance of the fish images, but for a surveyor who wants to take advantage of tablet technology for recording and saving data. Objective 4: Allow users to build their own surveys – this objective was removed from the project because it was determined to be out of the scope of the goal of Artedi. The app is designed to be a teaching tool, and certain functionality could hinder the ability to learn as you encounter fish. By allowing users to build and save a survey template, they could pre-populate an Artedi survey with fish that they expect

to find. This defeats the purpose of learning to associate the image of the fish with what you encounter because the act of searching for the fish among its conspecifics in the illustrated list subconsciously teaches you to associate other species with the shape and family. In this way, you learn to recognize other species while searching for a specific one.

Discussion

Mastering the skill of identifying fish requires years of practice and repetition . New surveyors are often burdened with referring to dense reference books before and after a series of dives in order to recall and retain what they observed. The impracticality of bringing these tomes underwater led to the development of smaller, fewer paged laminated guides that, albeit helpful, sacrificed coverage of many common reef fish species. Another potential pitfall is the use of real-life images which require a broad-side photo of the fish and an appropriate amount of lighting to illuminate the natural color/markings without over enhancing those features that respond well to the flash of the camera. To alleviate these stressors, the tablet application Artedi was developed in 2015 as a teaching tool for divers to bridge the gap between the classroom and underwater reality. Artedi transcribes the reference book into detailed illustrations of over 160 Caribbean reef fish and guides the user through a survey style template in which they are directed to first learn the family level grouping (or shape) to locate the appropriate species that they are observing underwater. This allows the user to document what they see while at the same time learning the taxonomic classifications of fish families and visually associating what they see with where that illustration is located within the app This process involves cognitive deciphering and consolidating information through visual processing. Continuous use and repetition encourages the learning of concepts and categories like “fish family” and individual species through a visual representation. Therefore, the use of technology as a facilitating tool, along with experience and repetition of the activity, will promote the association of information forming a more complete learning referred to as experiential learning (Kolb 1984) Not only this, but operant conditioning (Skinner 1938) with this tool yields the satisfaction of being able to recognize a species or family of fish and the repetition of said activity will serve as a reward which facilitates the formation of memory.

One goal of this project was to further advance the Artedi application to incorporate features that users requested during prototype testing and after its initial launch. In particular, the project transitioned to a web-based platform that allows any user with any type of tablet device to have access to the application. In fact, a user with a computer can access Artedi and use all its features without even being underwater (see Recommendations for how this can be useful). The addition of a data management system – MyArtedi – allows the user to access their surveys in the cloud and perform simple visual assessments and calculations. For example, when viewing a single survey, each species is presented with its density and abundance. The surveys can be exported for .csv for further processing. The user can also manipulate their data graphically, by selecting one or many surveys to see the species or family representation. Lastly, a map is available for the user to visualize where they perform their surveys. The GPS coordinates entered for each survey will automatically plot on the map, however in most instances, a survey will not

automatically locate the GPS coordinates unless the tablet has internet or cellular access. Unfortunately, the technology does not currently allow for a solution to this when not connected, which is most instances in the field. To overcome this, and likely the case in most instances of field work, the user just needs to record the GPS coordinates manually before they enter the water with the tablet, or they can add the coordinates afterwards on the MyArtedi system.

But perhaps the most unique and groundbreaking aspect of this project involved the experiment designed to test the effectiveness of Artedi at training new surveyors to identify fish. Not only did this require the application to be constantly utilized while underwater, thereby validating the ability to do so, but it also indicated some variables to consider in the learning process. The experiment exposed that the inherent differences among surveyors (i.e. skill level prior to the experiment, bias in observing certain species) introduces variability that could not be accounted for in the analysis. But, as hypothesized: 1) volunteer precision (measured as multivariate dispersion) increased with time and 2) similarity between untrained divers and experts increased with time. However, and contrary to our predictions, the overall speed at which those two processes occurred was not different between Artedi and Non-Artedi users.

Thus, we cannot ultimately conclude that Artedi was overall more effective at teaching fish identification, but we can conclude that Artedi was at the very least equally as effective overall and more effective for some individuals. Analyses of these individual cases indicated that this lack of differential response between Artedi and Non-Artedi users was due to the relative effect of prior knowledge that the volunteers had and the amount of study during the experiment, which was very heterogeneous among untrained divers (as evidenced by the quizzes results). Although efforts were made to construct a team of volunteers who were generally equal in experience, even the slightest familiarity with certain species or families obviously contributed to the results that we obtained. A surveyor that has some familiarity with fish (can identify a particularly common species like “blue tang”, or can recognize a family level group like “parrotfish”) introduces a bias into the learning process whereby they will prioritize what they already know when surveying and spend less time trying to associate and recognize those which they do not know. Furthermore, the hardware itself (iDive tablet housing, www.idivehousing.com) came with its own learning curve where a handling period affected the user’s ability to fully utilize the potential of Artedi. For example, Artedi divers had to be acclimated to the housing’s pressure controls – knowing when to insert or remove air as they changed depths in order to keep the plastic screen from touching the surface of the tablet – which may have affected the learning process initially.

However, the experiment ultimately illustrated the practical use of Artedi for those surveyors with no prior knowledge of fish identification or any prior bias of subconsciously prioritizing certain species over others. The two Artedi users without prior knowledge reached an experience level more closely resembling the expert at a faster rate than their Non-Artedi counterparts. This means that Artedi could accelerate the learning process, but certain prerequisites are required to achieve that result. Thus, Artedi can be very effective in training user groups, like citizen scientists, as most of these participants would have no prior knowledge of

fish identification and therefore be at the same starting point as two of the Artedi users from this study. A citizen science program could incorporate training with Artedi to facilitate a faster learning process and thereby achieving more reliable data.

Overall, this experiment resulted in training eight new surveyors who all improved their fish identification skills over the course of the project. At the end, each surveyor was more closely aligned to the expert than at the start of the experiment. This indicates that although Artedi was not overall more effective, it was more effective on a case by case basis and especially with individuals who did not have any knowledge about fish prior to the experiment. Furthermore, this experiment also indicated that Artedi did not hinder the ability to learn fish identification, which is also an important result showing that this technology provides a new, modern solution.

Recommendations

To build on this project, the PIs recommend creating a video using animations of the fish illustrations to create a known system whereby users can test their fish ID skills in a virtual classroom (or virtual field setting). This video would have a known number of individuals per species and a known number of species, so users could watch the video and use their computer or tablet to conduct a survey, and then see how accurate they were at the end. This would require an animator to create this video, but it should use the fish illustrations already created, if possible. Isla Mar is exploring this option. Additionally, we recommend an alternative hardware for the tablet and specifically a hardware designed for Android tablets. Isla Mar is exploring options and will consider approaching engineering programs and various schools to see if this idea could become a senior capstone project.

References

Cooper, C.B., Dickinson, J., Phillips, T., Bonney, R. 2007. Citizen science as a tool for conservation in residential ecosystems. Ecol Soc 12(2):11.

Dabbagh, N. and Kitsantas, A. 2012. Personal Learning Environments, social media, and selfregulated learning: A natural formula for connecting formal and informal learning. The Int High Ed 15(1):3-8.

Dickens, L.C., Goatley, C.H.R., Tanner, J.K., Bellwood, D.R. 2011. Quantifying relative diver effects underwater visual censuses. PLoS One 6:6 –8.

Dickinson, J.L., Zuckerberg, B., Bonter, D.N. 2010. Citizen Science as an Ecological Research Tool: Challenges and Benefits. Annu Rev Ecol Syst 41:149–72

Edgar, G.J., Barrett, N.S., Morton, A.J. 2004. Biases associated with the use of underwater visual census techniques to quantify the density and size-structure of fish populations. J Exp Mar Bio

Ecol 308:269–290

Harms-Tuohy, C.A., Schizas, N.V., Appeldoorn, R.S. 2016. Use of DNA metabarcoding for stomach content analysis in the invasive lionfish Pterois volitans in Puerto Rico. Mar Ecol Prog Ser 558:181-91.

Hrastinski, S. and Aghaee, N.M. 2012. How are campus students using social media to support their studies? An explorative interview study. Ed Inf Tech 17(4):451-464.

Kolb, D. 1984. Experiential learning: experience as the source of learning and development. Englewood Cliffs, NJ: Prentice Hall.

Nemeth, M. 2013. Distribution of roving herbivorous fishes on coral reefs at multiple spatial scales. PhD dissertation, University of Puerto Rico, Mayagüez, PR

Pittman, S.J., Hile, S.D., Jeffrey, C.F.G., Clark, R. and others. 2010. Coral reef ecosystems of Reserva Natural La Parguera (Puerto Rico): spatial and temporal patterns in fish and benthic communities (2001–2007). Tech Memo NOS NCCOS 107. NOAA, Silver Springs, MD

Skinner, BF. 1938. The Behavior of Organisms: An Experimental Analysis. New York: Appleton-Century.

Thompson, A.A. and Mapstone, B.D. 1997. Observer effects and training in underwater visual surveys of reef fishes. Mar Ecol Prog Ser 154:53-63.

Williams, I D , Walsh, W J , Tissot, B N , Hallacher, L E. 2006. Impact of observers’ experience level on counts of fishes in underwater visual surveys. Mar Ecol Prog Ser 310:185 –191.