GroIMP wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Structural QSM-based data analysis integrating R ======
Sometimes, you might want to conduct some statistical analysis on the structure of your simulated or measured plant outside of XL and GroIMP. GroIMP's query system is principally a very powerful tool for the data subsetting part of such analysis (finding parts of your structure that follow some topological relationships), but it can be hard to apply advanced statistical methods directly in GroIMP.

Here, we present an example on how to export structures from GroIMP and how to analyze them in R. There, the rTwig library allows for convenient analysis of QSM-like structures, which we will export from GroIMP.

Note that there are also the [[tutorials:grogra-functions|GROGRA Functions for structural analysis]]. These work directly in GroIMP and can provide similar outputs like rTwig, but they are not documented as well. Nonetheless, their wiki article here may give you some ideas on the types of analysis you can conduct, irrespective of the tools you use.

There are three typical data origins for structures that people want to analyze: Manually measured FASTRAK or [[groimp-platform:dtd-format|.DTD]] data and plants generated in GroIMP. This article covers all of these sources. There is also a fourth possible data source, which are QSMs. These can also be imported and analyzed in GroIMP. For example, GroIMP could be used to interactively correct QSM topologies.

===== Requirements =====
  * GroIMP "qsm" [[user-guide:pluginmanager|plugin]]
  * GroIMP "fastrakReader" plugin when working with FASTRAK data (use version 0.8 with GroIMP version 2.2)
  * Some recent version of R
  * R-library "rTwig" ([[https://github.com/aidanmorales/rTwig/|rTwig documentation on github]])

===== GroIMP import & export=====
As always in GroIMP, you can import your file using Object -> Insert File / Insert File to RGG. The difference is that if you insert to RGG, the file will be imported under the RGG Root instead of under Node.0. For our example here, always use "Insert File" (not under RGG), even though both can be used.
==== Import: FASTRAK ====
Importing FASTRAK data in .xml format requires the fastrakReader plugin.
Note that your data might be scaled in an unexpected way relative to your GroIMP project. In the picture below is an example of FASTRAK data inserted in a New RGG project, note the size of the FASTRAK tree relative to the "A" of New RGG.

{{:tutorials:fastrak-imported-new-rgg.png?400|}}

If you want to delete this structure, you can do Edit -> Clear Project Graph (which will remove everything, not only the thing you just imported).

Using the color() method below, you can color your measured FASTRAK data based on branch orders:
 
{{:tutorials:fastrak-orders.png?400|}}

As you can see, there is a possible measurement error here: One branch is colored in blue, which means it is of order 3, but it should probably be order 2. This is because the fastrakReader has to infer the topology of the structure based on the relative position of segments to each other. In this case, this segment happens to start closer to the other branch than to its actual mother segment:

{{:tutorials:error-branch.png?400|}}

<code java>
public void color (){
	[	(*f:F,(count((*f ((<)*<+)+F*))==0)*) ::> f.setShader(RED);    //all F of order 0
		(*f:F,(count((*f ((<)*<+)+F*))==1)*) ::> f.setShader(YELLOW); //all F of order 1]
		(*f:F,(count((*f ((<)*<+)+F*))==2)*) ::> f.setShader(GREEN);
		(*f:F,(count((*f ((<)*<+)+F*))==3)*) ::> f.setShader(BLUE);
		(*f:F,(count((*f ((<)*<+)+F*))==4)*) ::> f.setShader(BLACK);
]}
</code>

==== Import: DTD ====
DTD files can be imported in exactly the same way as FASTRAK files, but no special plugin is required.

==== Import: QSM ====
To import a QSM in GroIMP, you need to provide it as a comma-separated file with .qsm extension which contains these columns: start_x,start_y,start_z,end_x,end_y,end_z,id,parent,raw_radius,length,branch_order,branch. 
This is consistent with the rTwig naming convention. The intended usage pipeline is to standardize QSMs from various QSM tool origins in rTwig, then you can simply export the required variables from the $cylinder data as a table. 


==== Export ====
Export from GroIMP as .qsm is provided by the qsm plugin. Under View -> Export -> complete scene you need to specify a file name and select text/qsm as filetype. The resulting file is simply a comma separated table which contains information for every cylinder in your scene:
{{:tutorials:qsm-csv.png?1600|}}

You can also export to .qsm from RGG using:
<code java>
export3DScene(getWD() + my_name + ".qsm", new MimeType("text/qsm", null) , false );
</code>

===== Import in R =====
Please take a quick look the rTwig documentation to get an idea what is possible with this library and how it works in general. The main point here is that we can import our .qsm file using the reconstruct_qsm() function:

<code sas>
library(rTwig)
qsm <- reconstruct_qsm(
  cylinder = read.csv("path/to/my/qsm.qsm"),
  id = "id", parent = "parent", radius = "raw_radius",
  branch_order = "branch_order", # this is important to retain original branch order defined in GroIMP!
  start_x = "start_x", start_y = "start_y", start_z = "start_z",
  end_x = "end_x", end_y = "end_y", end_z = "end_z"
)
plot_qsm(qsm, color = "branch_order", palette = "viridis")
</code>

{{:tutorials:rtwig-importplot.png?400|}}

===== Analysis in R =====
The most useful rTwig functions are probably the plot_qsm() function you already saw (it has diverse coloring options and can also be used for point clouds) and the tree_metrics() function. Besides this, you can always just use the qsm cylinder data to come up with your own analysis. tree_metrics() is somewhat similar to the GROGRA functions, it outputs a whole range of overall descriptors and distributions for various aspects of your tree's morphology. Pleas reference the rTwig documentation for further details.

<code sas>
metrics <- tree_metrics(qsm)
plot(metrics$stem_taper$height_m, metrics$stem_taper$diameter_cm,
     xlab = "Height [m]",
     ylab = "Diameters [cm]")
lines(metrics$stem_taper$height_m, metrics$stem_taper$diameter_cm)
</code>

{{:tutorials:stem_taper_231_5.png?400|}}

tree_metrics() provides some useful aggregations that are similar to GroIMP queries, like for example the aggregation of individual cylinder segments into branches.
The actual analysis to perform will however heavily depend on the type of your data source and how the data was collected (e.g. biologically meaningful shoot-based data vs. pure geometry acquisitions). Supposing you have Fastrak data where every cylinder segment represents a yearly shoot, you could e.g. analyze the yearly apical growth of the stem on several spruce trees like this, just using the raw QSM data:

<code sas>
library(dplyr)
library(ggplot2)

folder_path <- "D:/groimp/tiny_trees_analysis_example/f_qsms"
file_list <- list.files(path = folder_path, pattern = "\\.qsm$", full.names = TRUE)
qsm_list <- list()
for (file in file_list) {
  cyl_data <- read.csv(file)
  tree_id <- tools::file_path_sans_ext(basename(file))
  qsm <- reconstruct_qsm(
    cylinder = cyl_data,
    id = "id", parent = "parent", radius = "radius",
    branch_order = "branch_order",
    start_x = "start_x", start_y = "start_y", start_z = "start_z",
    end_x = "end_x", end_y = "end_y", end_z = "end_z") %>% 
    mutate(tree_id = tree_id)
  qsm_list[[tree_id]] <- qsm
}

stem_list <- lapply(qsm_list, function(qsm){
  qsm %>% 
    filter(branch_order == 0) %>%
    arrange(base_distance) %>% 
    mutate(year = 1:nrow(.))})

all_stems <- bind_rows(stem_list)
all_years <- sort(unique(all_stems$year))
all_stems %>%  
  ggplot(., aes(x = year, y=length)) +
    geom_point(aes(color = tree_id)) + 
    geom_smooth(method = "lm") +
    ggtitle("Yearly apical shoot lengths") +
    xlab("Year") + ylab("Shoot length [m]") + 
    scale_x_continuous(breaks = all_years) +
    theme_minimal()
</code>

{{:tutorials:yearly_shoots_tiny_spruce.png?400|}}