high-throughput phenotying

Beyond counting: computer science partnership helps speed up plant science experiments

May 12, 2021 By Lauren Hines in Research Tags: BIPS, computer science, high-throughput phenotying, plant science

Janlo Robil, graduate student in the Paula McSteen lab, came up with the GrasVIQ project after he finished a project that required him to count hundreds of plant leaf veins. | photo by Lauren Hines, Bond LSC

By Lauren Hines | Bond LSC

It’s not surprising that researchers feel discouraged when pursuing projects that involve plant leaf vein density analysis. Manually counting individual leaf veins and measuring their density to understand how nutrients are transported in plants can take weeks of tedious work.

That’s how Janlo Robil was feeling when he was working on a maize leaf project while rotating through the Paula McSteen lab in 2016 in Bond LSC.

Now, researchers can cut that time down to just minutes.

As part of the Bioinformatics in Plant Sciences (BIPS) program, undergraduate and graduate students in plant sciences and computer science have come together to create the first phenotype image analysis system for monocot plant leaf veins called, GrasVIQ.

The GrasVIQ software can process photos of grass leaves, automatically detect and count vertical grass leaf veins and calculate various quantitative measurements such as vein density, width and separation. These measurements help analyze leaf vein network patterns.

“The time that you use for collecting data, you can already use for analysis,” said Janlo Robil, head of the project and graduate student in the McSteen lab. “It will also help you in your decision about the direction of your research because the faster you get the data, then the better idea you have if you still need to pursue this or not.”

The project was spurred by Robil in 2017 when he went to a seminar by Filiz Bunyak, assistant research professor in the Electrical Engineering and Computer Science Department.

Bunyak was presenting her group’s interdisciplinary image processing and computer vision research, including their work on plant phenotype analysis. Robil got the idea to apply it to plant leaf veins.

“It started from a simple question,” said Ke Gao, co-author and graduate student in the Electrical Engineering and Computer Science Department. “Can we help them do their analysis faster with repeatable quantified results? We were hopeful we could use our previous experiences in plant image analysis to help Janlo and his lab conduct their analysis more efficiently.”

However, the project was on hold until 2019. Robil simply didn’t have the time or manpower for this project until he heard about the BIPS program.

“I think [the BIPS program is] a very good way of promoting interdisciplinarity in research,” Robil said. “Had I not asked the computer science people to collaborate with me, I would still be manually counting veins right now, and not using software to quantify veins more quickly. I think interdisciplinarity can move science forward and faster, literally.”

The software can identify the vertical veins with 95% accuracy and 92% accuracy for transverse veins when compared to the manual quantification.

“Without any doubt, we believe that the software can do a better job by just the rate of it,” Robil said.

The GrasVIQ phenotyping image analysis software can classify and quantify stained veins of grass and corn leaves. doi.org/10.1111/tpj.15299

While the software saves plant science researchers time and energy, it also provides computer science students something as well.

“Collaborating with plant scientists gives us the opportunity to leverage our expertise in engineering, algorithm development, and mathematical models to contribute to the solution of a real-world problem,” Gao said.

BIPS pairs undergraduate and graduate students from plant sciences and computer science together to work on a project. For undergraduate Claire Neighbors in the McSteen lab and computer science undergraduate Michael Boeding, this was their first published paper.

“It’s super exciting,” Neighbors said. “It feels good like all my hard work meant something.”

Neighbors was the one who had to create the images and do the manual counting to compare to the software’s counting. She said the process took her months.

“It was worth something so that we could make this software valid,” Neighbors said. “Someone had to put in the hours to be able to make those ground truths to show that this software works. So, it feels really good that I was able to contribute something.”

While the software has much to offer, the researchers believe it could be better. Robil thinks vein classification can be improved using machine learning even though the software achieves a detection accuracy of more than 90% for vertical veins.

The current software is designed to be used by scientists and does not have a very friendly user interface. Building a better user interface and further improving the analytics using machine learning approaches are future goals for Gao. GrasVIQ will enable generations of training data for these machine learning-based improvements.

Nonetheless, the software provides an escape from endlessly counting veins.

“We learned how to quantify the veins quickly, but like a year from now, that isn’t going to be what’s taking us months to do,” Neighbors said. “It’s going to be figuring out why this gene made the veins look weird or produced less or thinner, thicker, which is going to make things more efficient in the future.”

More information about the project can be found in, “GrasVIQ: An Image Analysis Framework for Automatically Quantifying Vein Number and Morphology in Grass Leaves.” The paper was published on April 29 in The Plant Journal. The study was done by Janlo Robil, Ke Gao, Claire Neighbors, Michael Boeding, Francine Carland, Filiz Bunyak and Paula McSteen. The Bioinformatics and Plant Sciences (BIPS) program is funded under the National Science Foundation, Cellular Dynamics and Function MCB-1818312 to David Mendoza-Cozatl (University of Missouri, Columbia).

Connecting the World Through the Cloud

December 3, 2020 By Becca Wolf in Research Tags: bioinformatics, COVID-19, high-throughput phenotying

Maria Clare Lusardi, an undergraduate student in the Mendoza lab, uses the cloud, a program apart of CyVerse. | photo by Becca Wolf, Bond LSC

By Becca Wolf | Bond LSC

Clouds come in many shapes and sizes. Some are big and fluffy, others dark and ominous.

Or, as in David Mendoza’s case, the cloud is a hub of experiment information.

Mendoza, an associate professor of plant sciences and scientist in Bond Life Sciences Center, recently joined the CyVerse, a cyberinfrastructure system used by Mendoza’s lab that acts as and allows his team and collaborators to see data in real time. CyVerse, funded by the National Science Foundation, allows life science researchers to share and store their research data in the cloud.

This sort of system has come in quite handy for Mendoza as the Covid-19 pandemic limited the number of people allowed in a lab last spring, forcing him to look for a better way to work remotely but still together.

“These difficulties make you more creative. Covid-19 is a serious, terrible thing, and I could tell my lab was frustrated because they wanted to continue working, but couldn’t because of the restrictions,” Mendoza said. “We needed this. We started implementing the cloud and the opportunities to keep working and collaborate continued and worked out nicely.”

Mendoza connected the cloud to the robots his lab uses to take photos of experiments. These photos are sent every other hour to the cloud, so Mendoza and collaborators can check in on experiments from anywhere in the world.

To get connected, Mendoza brought in Drew Dahlquist, an undergraduate student, to help.

“As a computer science student, the stuff I’m doing feels very natural, but I didn’t realize how much other areas of science could use it. Before I came on, they were storing all the images just on a laptop in the lab and sending them around on thumb drives, which can be very inefficient,” Dahlquist said. “It’s going to greatly improve what David’s doing in the lab.”

Dahlquist has been transferring Mendoza’s data from the thumb drives to the cloud, a process he plans on being done with by the end of the semester.

“Hopefully, I can start tweaking it a little more to refine it and then just see what other needs pop up in the future,” Dahlquist said.

The Project

The main task benefitting from this new approach is a high-throughput phenotyping project, led by Landon Swartz, an undergraduate student in his lab.

“I was kicked off campus when we had quarantine last semester, so all of my projects stopped suddenly,” Swartz said. “I was just stuck at home doing remote stuff, which was not helpful. So that’s where Drew came in to do the cloud stuff. We realized that this is going to be a regular thing for the next few years and will likely happen again in the future.”

The cloud helps Swartz and Mendoza because instead of waiting a week to see how an experiment is doing, it receives frequent updates, so they can make changes to an experiment in real time instead of after the fact.

This is beneficial when they observe how leaf color changes when over or underexposed to nutrients like iron. This can cause chlorosis — when the leaves turn yellow from lack of iron and die. The cloud allows them to see if chlorosis is occurring at a different rate than they expected. With this information, they can change the parameters and continue the experiment without any issues.

“We are establishing protocols where first thing in the morning the machine is going to send me an email with the first photo of the day and to say that the robots are okay. Then, at night before bed, I’ll get the last picture of the day,” Mendoza said.

The camera used to take photos of experiments in the Mendoza lab. These photos are then sent to the cloud. | photo by Becca Wolf, Bond LSC

Collaboration made Easy

In addition to making experiments more accessible for the Mendoza lab, the cloud allows the lab to easily share data with collaborators.

“We started to realize that having our phenotyping robots be controlled remotely or be able to put our data onto the cloud would be super useful, not just for our lab, but also for labs around the world,” Swartz said. “So, maybe the U.S. goes into quarantine, but we have a sister lab in South Africa that is able to pick up our data where we left off and continue to run experiments for us.”

Having multiple labs look at the data at the same time helps create more ideas and exposes more people to those ideas.

“Our goal is to put the data out there and let more people try different approaches to analyze the data so that all of us as a field, can decide what is the best way to do standardized experiments,” Mendoza said.

Also, with the sister lab on another continent, there are people who can check in on the experiments around the clock.

“For instance, the lab in South Africa can see the data in real time. They can see if we need to change the light, increase the light, or make the night longer, and they could also control the nutrients among other things,” Mendoza said.

The cloud will only help experiments in the future.

“The internet is a great resource to collaborate. It’s not often used like that, but we can put data that we’re getting in the moment into this huge database that anyone across the world can look at. It really changes the whole game of science,” Swartz said. “It’s no longer about who can publish things first, it has really become an everyone’s in it together kind of thing. And that’s where I really would like to see this go, and especially in the next 10 or 20 years.”

While it started because of a pandemic, Mendoza sees it as a useful tool regardless of lab restrictions.

“Now that we’ve seen the possibilities, I don’t think we’re going back just because of the amount of experiments that you can do,” Mendoza said. “We actually plan on building more machines with more sensors, so it’s not going away anytime soon.”

The future of lab technology

October 4, 2017 By Samantha Kummerer in Research Tags: collaboration, computer scientists, high-throughput phenotying, plant science, technology

Computer scientists create applications to speed up research in the lab

Ph.D. student Ke Gao and computer scientist Filiz Bunyak collaborate with researchers at the Bond Life Sciences Center. The pair helps advance high-throughput phenotyping by developing applications and algorithms for image analysis. | Photo by Samantha Kummerer, Bond LSC

By Samantha Kummerer, Bond LSC

Three years ago, Ke Gao stood uncomfortably beside rows of biomedical students and plant scientists at the Bond Life Sciences research fair. His poster wasn’t discussing the DNA of seeds or how plants transport nutrients but rather a scientific device.

“At the beginning, the visitors didn’t understand what we were presenting, but once I explained how our application can help them accelerate their research and how we can really turn their phones into a research device, they got really excited,” Gao explained.

Gao’s presentation highlighted a mobile app that transforms images of seeds into objective, quantitative data.

It started with a simple problem. Plant scientists were manually comparing hundreds and in some cases thousands, of seed photos. The process was meticulous, slow and subjective.

The solution began with a collaboration with Michele Warmund (Plant Sciences), Tommi White (MU Electron Microscopy Core) and Filiz Bunyak (Computer Science) that led to a MU Interdisciplinary Innovations Fund grant.

Gao was part of this team that developed an algorithm to turn the photos of seeds from the field into data with the touch of the button.

Gao explained the app is very similar to Instagram.

A user takes or uploads photos of seeds. Then the app calculates measurements describing shape, color and size characteristics of the seeds. This data can be emailed or stored in a database.

Some experiments need thousands of seeds analyzed; this would be a massive feat for even a group of students. With this app, hundreds of seeds can be photographed and measured from a single photo. The app analyzes each seed individually and also computes measurement averages for groups of seeds.

There are other apps that analyze seeds, but this is the first mobile application as far as the team knows. Its ability to analyze multiple seeds at once, even if they are touching is also an outstanding ability. Bunyak’s previous experience developing applications to quantify microscopy images and videos of touching and clumping cells helped them design the algorithm to make that function possible.

This isn’t just a problem for researchers in this one lab or even at the University of Missouri.

MU Computer Science professor, Filiz Bunyak, said noninvasive methods to observe and understand biology, imaging equipment and corresponding computing devices have advanced considerably in recent years, leading scientists to produce large amounts of data. The ability for researchers to analyze and quantify this large amount of complex and unstructured data, however, was still missing. Bunyak said this app began as a project to advance scientists’ capabilities to automatically analyze image-based plant phenotyping.

Further collaboration

Bunyak and her students are advancing the field of high-throughput phenotyping beyond this mobile app.

High-throughput phenotyping (HTP) refers to the process of connecting an organism’s DNA makeup to its physical characteristics; it is also a hot topic buzzing through the science community in the last five years.

Two years ago, Bond LSC scientist David Mendoza, who studies how plants collect nutrients, said he never imagined he would be doing HTP.

“The old way of doing this is growing plants on plates and, I’m not kidding, with a ruler you measure how long the roots are,” Mendoza explained of the traditional process that now seems archaic.

Now, the lab is working with computer scientists to design a robot to code the measurements for multiple roots at a single time. For a student, it would take 15 minutes, but now it’s complete in an instant.

Speed isn’t the only reward researchers are reaping.

Bunyak said computational image analysis allows researchers to come up with new ways to quantify and study data that they were not even able to do before, leading to the design of novel experimental methods.

Ruthie Angelovici is another Bond LSC researcher who uses computer scientists to aid in her research.

She said without computer imaging there would be no way for her team to do research that measures plants physical and biochemical traits. Angelovici’s lab uses Bunyak’s mobile app system but on a computer. Eight plants are photographed at once and the application keeps track of features of plants such as shape, color and area as they develop.

What is really revolutionary to Angelovici is the ability for the data of plant growth parameters to be stored and revisited without the need to re-grow. This contrasts with past experiments where researchers would scribble some notes and never be able to return.

“It’s not lost and I think that’s a big step in this field,” Angelovici said.

The collaboration is creating more than advanced tools by fostering a new way to think and approach research.

Rather than buying pre-existing software, the groups from Bond LSC utilizes the resources on campus to build their own devices.

“I would have been in front of a black box that is doing things for me and that would not have given me the tools to teach to my students,” Mendoza reflected. “Now I know what they need to learn to be competitive. Now I know what the gaps are and how they can be filled. I think that was worth it.”

Mendoza’s team publishes all the instruction to its robot online, so the technology can aid other labs in making faster discoveries at a lower price.

Angelovici compared it to buying a cake versus making a cake — at the end of the creation process, she said she would have the knowledge to do a lot of other experiments.

This new way of thinking already began to pay off this summer when her lab expanded computer software to analyze seed size.

“We only approached it because we saw how things worked together. I just pitched a project to engineering about seed collector. Again, this opened my eyes that even undergraduates can do something not so difficult for engineers, but I have no clue how to do it,” Angelovici said.

Mendoza agreed the collaboration is exciting but challenging, “You got a Ph.D. and you got a faculty position and you think you know stuff. When I started this I realized how much I don’t know, but at the same time it reminded me that it is really cool to learn something new.”

Both teams continue to work towards maximizing the functions of their individual machines, but even after the projects reach fruition the collaboration will not be over.

“On the contrary, I think we’re going to keep building more and more and better,” Mendoza said.

Nowadays, Gao no longer feels out of place at the Life Sciences fairs. Researchers from various labs come up to him and ask how they can implement his app in their own lab.

“It seems like I’m doing something that can really help people, so that’s the best part of this process,” he said.

Ruthie Angelovici is an assistant professor in the Division of Biological Sciences and is a researcher at Bond Life Sciences Center. She received her degrees in plant science from institutions in Israel — her B.S. and M.S. from Tel Aviv University, and her Ph.D. from the Weizmann Institute of Science in Rehovot. She was a postdoctoral fellow at the Weizmann Institute and at Michigan State University and has been at MU since fall of 2015.

David Mendoza is an associate professor in Plant Sciences, Life Sciences Center investigator and a member of the Interdisciplinary Plant Group. His research focuses on the mechanisms plants use to resist toxic elements or acquire nutrients. He received his Ph.D. in biochemistry from UNAM in Mexico City and continued on to do post-doc training at UC San Diego.

Filiz Bunyak is an assistant research professor in the Department of Computer Science. She received her bachelors and masters degree from Istanbul Technical University and her Ph.D. from the University of Missouri- Rolla. Her work focuses on computer imaging, image processing, and biomedical image analysis.

Ke Gao is a doctoral student in the University of Missouri’s Department of Electrical Engineering and Computer Science. He earned his bachelor’s of science from the Henan University of Science and Technology in China.