{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Getting Started with Exploring Segmentations "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Before you start!\n",
    "\n",
    "- This notebook assumes that shapeworks conda environment has been activated using `conda activate shapeworks` on the terminal.\n",
    "- See [Getting Started with Notebooks](getting-started-with-notebooks.ipynb) for information on setting up your environment and notebook shortcuts. \n",
    "- See [Getting Started with Segmentations](getting-started-with-segmentations.ipynb) to learn how to load and visualize binary segmentations.\n",
    "- Note example output was generated on Linux/Mac environment and may look different on Windows.\n",
    "\n",
    "## In this notebook, you will learn:\n",
    "\n",
    "1. How to define your dataset location and explore what is available in it\n",
    "2. How to explore your dataset\n",
    "3. How to decide the grooming pipeline needed for your dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import shapeworks as sw"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Defining and exploring your dataset\n",
    "\n",
    "### Defining dataset location\n",
    "\n",
    "You can download exemplar datasets from [ShapeWorks data portal](https://girder.shapeworks-cloud.org) after you login. For new users, you can [register](https://girder.shapeworks-cloud.org/#?dialog=register) an account for free. Please do not use an important password.\n",
    "\n",
    "After you login, click `Collections` on the left panel and then `use-case-data-v2`. Select the dataset you would like to download by clicking on the checkbox on the left of the dataset name. See the video below.\n",
    "After you download the dataset zip file, make sure you unzip/extract the contents in the appropriate location.\n",
    "\n",
    "**This notebook assumes that you have downloaded `ellipsoid_1mode` and you have placed the unzipped folder `ellipsoid_1mode` in `Examples/Python/Data`.** Feel free to use your own dataset.  \n",
    "\n",
    "\n",
    "<p><video src=\"https://sci.utah.edu/~shapeworks/doc-resources/mp4s/portal_data_download.mp4\" autoplay muted loop controls style=\"width:100%\"></p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#import relevant libraries\n",
    "import os\n",
    "import pyvista as pv\n",
    "import numpy as np\n",
    "\n",
    "# dataset name is the folder name for your dataset\n",
    "datasetName  = 'ellipsoid_1mode'\n",
    "\n",
    "# path to the dataset where we can find shape data \n",
    "# here we assume shape data are given as binary segmentations\n",
    "shapeDir      = '../../Data/' + datasetName + '/segmentations/'\n",
    "    \n",
    "print('Dataset Name:     ' + datasetName)\n",
    "print('Shape Directory:  ' + shapeDir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### What is available in the dataset?\n",
    "\n",
    "First let's see how many shapes we have in the dataset.\n",
    "\n",
    "**File formats:** For binary segmentations, all [itk-supported image formats](https://insightsoftwareconsortium.github.io/itk-js/docs/image_formats.html) can be used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import glob \n",
    "from pathlib import Path \n",
    "\n",
    "\n",
    "shapeExtention = '.nrrd'\n",
    "\n",
    "# let's get a list of files for available segmentations in this dataset\n",
    "# * here is a wild character used to retrieve all filenames \n",
    "# in the shape directory with the file extensnion\n",
    "shapeFilenames = sorted(glob.glob(shapeDir + '*' + shapeExtention)) \n",
    "\n",
    "print ('Number of shapes: ' + str(len(shapeFilenames)))\n",
    "print('Shape files found:')\n",
    "for shapeFilename in shapeFilenames:\n",
    "    shapeFilename = Path(shapeFilename)\n",
    "    print(shapeFilename)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Exploring your dataset\n",
    "\n",
    "We would like to better understand the given dataset to decide the appropriate grooming (preprocessing) pipeline/step to prepare it for shape modeling."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loading your dataset\n",
    "\n",
    "First step is to load the dataset. \n",
    "\n",
    "**Note:** If your dataset is large (large volumes and/or large number of segmentations), you could select a subset for this exploration step."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# list of shape segmentations\n",
    "shapeSegList = []\n",
    "\n",
    "# list of shape names (shape files prefixes) to be used \n",
    "# for saving outputs and visualizations\n",
    "shapeNames   = [] \n",
    "\n",
    "# loop over all shape files and load individual segmentations\n",
    "for shapeFilename in shapeFilenames:\n",
    "    print('Loading: ' + shapeFilename)\n",
    "    \n",
    "    # current shape name\n",
    "    segFilename = shapeFilename.split('/')[-1] \n",
    "    shapeName   = segFilename[:-len(shapeExtention)]\n",
    "    shapeNames.append(shapeName)\n",
    "    \n",
    "    # load segmentation\n",
    "    shapeSeg = sw.Image(shapeFilename)\n",
    "    \n",
    "    # append to the shape list\n",
    "    shapeSegList.append(shapeSeg)\n",
    "\n",
    "num_samples = len(shapeSegList)\n",
    "print('\\n' + str(num_samples) + \n",
    "      ' segmentations are loaded for the ' + datasetName + ' dataset ...')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualizing your dataset\n",
    "\n",
    "Now let's visualize all samples in a grid using `pyvista`. You may need to call `pv.close_all()` every once in a while to clean up the unclosed plotters.\n",
    "We will use `sw.plot_volumes` function from the Shapeworks python module. This function will take in a list of shapeworks images as input and initiate a pyvista plotter to render multiple windows, each with a single segmentation, add segmentations to the plotter, and start rendering."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define parameters that controls the plotter\n",
    "use_same_window = False # plot using multiple rendering windows if false\n",
    "notebook        = False # True will enable the plots to lie inline\n",
    "show_borders    = True  # show borders for each rendering window\n",
    "shade_volumes   = True  # use shading when performing volume rendering\n",
    "color_map       = \"viridis\" # color map for volume rendering, e.g., 'bone', 'coolwarm', 'cool', 'viridis', 'magma'\n",
    "show_axes       = True  # show a vtk axes widget for each rendering window\n",
    "show_bounds     = True  # show volume bounding box\n",
    "show_all_edges  = True  # add an unlabeled and unticked box at the boundaries of plot. \n",
    "font_size       = 10    # text font size for windows\n",
    "link_views      = True  # link all rendering windows so that they share same camera and axes boundaries\n",
    "\n",
    "# plot all segmentations in the shape list\n",
    "sw.plot_volumes(shapeSegList,    \n",
    "             volumeNames     = shapeNames, \n",
    "             use_same_window = use_same_window,\n",
    "             notebook        = notebook,\n",
    "             show_borders    = show_borders,  \n",
    "             shade_volumes   = shade_volumes, \n",
    "             color_map       = color_map,\n",
    "             show_axes       = show_axes,  \n",
    "             show_bounds     = show_bounds,\n",
    "             show_all_edges  = show_all_edges, \n",
    "             font_size       = font_size,   \n",
    "             link_views      = link_views\n",
    "             )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p><video src=\"https://sci.utah.edu/~shapeworks/doc-resources/mp4s/nb-explore-vols.mp4\" autoplay muted loop controls style=\"width:100%\"></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Deciding the grooming pipeline needed for your dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Does this dataset need grooming? What are grooming steps needed? Let's inspect the segmentations. What do we observe?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Voxel spacing\n",
    "\n",
    "Voxel spacing are not isotropic, i.e., voxel size in each of the three dimensions are not equal.This can be identified if you zoom-in in the visualization below and observe the step size in the x,y,z direction. Anisotropic spacing could adversely impact particles optimization since shapeworks assumes equal voxel spacing. Some datasets might also have different voxel spacings for each segmentation. \n",
    "\n",
    "*Hence, it is necessary to bring all segmentations to the same voxel spacing that is equal in all dimensions.* \n",
    "\n",
    "Another observation is voxel spacing is relatively large. This can be observed by the pixelated volume rendering and the jagged isosurface. \n",
    "\n",
    "*We can improve the segmentation resolution by decreasing voxel spacing.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pyvista as pv\n",
    "# to better appreciate the pixelated nature of these segmentations, we need to only visualize \n",
    "# the binary segmentation, notice the thick slices\n",
    "\n",
    "shapeIdx = 10\n",
    "shapeSeg = shapeSegList[shapeIdx]\n",
    " \n",
    "\n",
    "shapeSeg_vtk = sw.sw2vtkImage(shapeSeg, verbose = True)\n",
    "\n",
    "sw.plot_volumes(shapeSeg_vtk)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Segmentations and image boundaries \n",
    "\n",
    "Some segmentations are very close to the image boundary, not leaving enough room for particles (correspondences) to move and spread over these surface regions. In particular, particles could overshoot outside the image boundary during optimization. \n",
    "\n",
    "Furthermore, if a segmentation touches the image boundary, this will result in an artificially (i.e., not real) open surface. \n",
    "\n",
    "*Hence, these segmentations needs to be padded with background voxels (zero-valued) to create more room along each dimension.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's inspect a segmentation that touches the image boundaries\n",
    "shapeIdx = 13\n",
    "shapeSeg = shapeSegList[shapeIdx]\n",
    "\n",
    "shapeSeg_vtk = sw.sw2vtkImage(shapeSeg, verbose = False)\n",
    "\n",
    "sw.plot_volumes(shapeSeg_vtk)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p><img src=\"https://sci.utah.edu/~shapeworks/doc-resources/pngs/shape_boundary.png\"></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "This segmentation touches the image boundary and hence will result in an artificially open surface. To inspect this behavior, we need extract a surface mesh (isosurface) from each segmentation. An isosurface is a three-dimensional surface that represents points of a constant value (aka isovalue) within the given volume of space."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's see if there's a function that extracts an isosurface from an image\n",
    "# use dot-tap to get a list of functions/apis available for shapeSeg\n",
    "\n",
    "# found it - toMesh, let's see its help\n",
    "help(shapeSeg.toMesh)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `toMesh` function needs an isovalue, which is the constant value the represents the surface of interest. Since a shape segmentation is a binary image, the foreground is expected to have the value of 1 (white) and the background should have a zero value (black), so an appropriate isovalue to extract the foregound-background interface a value in between, e.g., 0.5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "# let's make sure that our assumptions about the voxel values are correct\n",
    "# is the given volume a binary segmentation?\n",
    "\n",
    "# first convert to numpy array\n",
    "shapeSeg_array = shapeSeg.toArray()\n",
    "\n",
    "# make sure that it is a binary segmentation\n",
    "voxelValues = np.unique(shapeSeg_array)\n",
    "print('\\nVoxel values:' + str(voxelValues))\n",
    "\n",
    "if len(voxelValues) > 2:\n",
    "    print('WARNING: ' + shapeName + ' is not a bindary segmentation. Voxels have more than two distinct values')\n",
    "    print('PLEASE make sure to use binary segmentations')\n",
    "else:\n",
    "    print('Shape ' + shapeName + ' is a binary segmentation')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# now define the isovalue, in case a binary segmentation has a foreground label that is not 1\n",
    "# we need to obtain a value inbetween\n",
    "\n",
    "# get min and max values\n",
    "minVal = shapeSeg_array.min()\n",
    "maxVal = shapeSeg_array.max()\n",
    "\n",
    "print('\\nMinimum voxel value: ' + str(minVal))\n",
    "print('Maximum voxel value: ' + str(maxVal))\n",
    "\n",
    "isoValue = (maxVal - minVal)/2.0\n",
    "print('\\nisoValue = ' + str(isoValue))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's extract the segmentation isosurface and visualize it\n",
    "\n",
    "# extract isosurface\n",
    "shapeMesh = shapeSeg.toMesh(isovalue = isoValue)\n",
    " \n",
    "# sw to vtk\n",
    "shapeMesh_vtk = sw.sw2vtkMesh(shapeMesh)\n",
    "\n",
    "sw.plot_meshes([shapeMesh_vtk])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p><img src=\"https://sci.utah.edu/~shapeworks/doc-resources/pngs/explore_seg.png\"></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So, we have been able to extract a segmentation's isosurface and visualize it as a surface mesh. It is worth noting that the jagged surface is due to the anisotropic voxel space (with spacing in z-dimension is double that of x- and y-dimensions) and large voxel size."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Shape alignment\n",
    "\n",
    "One can observe from the segmentation visualization that they are not roughly aligned, i.e., they do not share the same coordinate frame where each individual shape is located differently compared to other shapes. \n",
    "\n",
    "*Aligning shapes is a critical preprocessing step to avoid the shape model to encode variabilities pertaining to global transformations such as rotation and translation.* "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's inspect some segmentations where we can observe misalignment\n",
    "shapeIdxs = [8,9,10]\n",
    "\n",
    "shapeSegSubset   = [shapeSegList[shapeIdx] for shapeIdx in shapeIdxs ]\n",
    "shapeNamesSubset = [shapeNames[shapeIdx]   for shapeIdx in shapeIdxs ]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To inspect how mutliple segmentation are spatially aligned with respect to each other, we will visualize their surfaces in the same rendering window. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "shapeSegIsosurfaces     = []\n",
    "shapeSegIsosurfaces_vtk = []\n",
    "\n",
    "for shapeSeg in shapeSegSubset:\n",
    "    \n",
    "    # extract isosurface\n",
    "    shapeIsosurface = shapeSeg.toMesh(isovalue = isoValue)\n",
    "    \n",
    "    shapeSegIsosurfaces.append(shapeIsosurface)\n",
    " \n",
    "    # sw to vtk\n",
    "    shapeSegIsosurfaces_vtk.append(sw.sw2vtkMesh(shapeIsosurface, verbose = False))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sw.plot_meshes(shapeSegIsosurfaces,       \n",
    "            use_same_window = True, \n",
    "            notebook        = False,  \n",
    "            show_borders    = True,  \n",
    "            meshes_color    = ['tan', 'blue','red'], \n",
    "            mesh_style      = \"surface\", \n",
    "            show_mesh_edges = False, \n",
    "            show_axes       = True,  \n",
    "            show_bounds     = True,  \n",
    "            show_all_edges  = True,  \n",
    "            font_size       = 10,    \n",
    "            link_views      = True   \n",
    "           )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p><video src=\"https://sci.utah.edu/~shapeworks/doc-resources/mp4s/nb-explore-iso-align-pv.mp4\" autoplay muted loop controls style=\"width:100%\"></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Too much background\n",
    "\n",
    "Image boundaries are not tight around shapes, leaving irrelevant background voxels that might increase the memory footprint when optimizing the shape model. \n",
    "\n",
    "*We can crop segmentations to remove unnecessary background.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "shapeIdx = 12\n",
    "shapeSeg = shapeSegList[shapeIdx]\n",
    "\n",
    "shapeSeg_vtk = sw.sw2vtkImage(shapeSeg, verbose = False)\n",
    "\n",
    "sw.plot_volumes(shapeSeg_vtk)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p><img src=\"https://sci.utah.edu/~shapeworks/doc-resources/pngs/explore_background.png\"></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Binary segmentations\n",
    "\n",
    "In general, this binary representation is not useful for finite numerical calculation of surface geometry and features that are required in shape modeling, which assumes the image is a sampling of a smooth function. \n",
    "\n",
    "Hence, ShapeWorks makes use of the signed distance transform of the binary segmentation that does satisfy this criterion. \n",
    "\n",
    "*For the correspondence optimization step, shapes can be represented as the zero level set of a smooth signed distance transform.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tentative grooming\n",
    "\n",
    "Hence, a tentative grooming pipeline entails the following steps:   \n",
    "1. Resampling segmentations to have smaller and isotropic voxel spacing   \n",
    "2. Rigidly aligning shapes   \n",
    "3. Cropping and padding segmentations   \n",
    "4. Converting segmentations to smooth signed distance transforms   \n",
    "\n",
    "\n",
    "Let the fun begins!!! Please visit [Getting Started with Grooming Segmentations](getting-started-with-grooming-segmentations.ipynb) to learn how to groom your dataset."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.8"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": true,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
