{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# FCIDECOMP software usage examples"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebooks presents examples of the different usage patterns of the FCIDECOMP software."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The FCIDECOMP software enable users to read NetCDF files compressed with the JPEG-LS compression filter. Once installed, it can be used with:\n",
    "\n",
    "- Python libraries, such as ``xarray`` adn ``netcdf4``\n",
    "- netCDF-C based CLI tools, such as ``nccopy``\n",
    "- The EUMETSAT Data Tailor, through the ``epct_plugin_mtg`` plugin\n",
    "- Some netCDF-Java based applications, such as Panoply\n",
    "\n",
    "In the following sections, usage examples of the first three points above are reported. For usage with NetCDF-Java based applications, see the README file.\n",
    "\n",
    "In order to run the following examples, ensure that the ``HDF5_PLUGIN_PATH`` is set to the path of the directory containing the FCIDECOMP software libraries, and that the ``EPCT_TEST_DATA_DIR`` is set as described in the ``README`` accompanying this notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## General imports and setting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the following, general purpose Python libraries and the ``fcidecomp`` Python module are loaded\n",
    "\n",
    "**NOTE**: if the FCIDECOMP software has been correctly installed and the ``HDF5_PLUGIN_PATH`` has been set to the right location, no error should be raised"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following line is needed to show plots in the notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's assign to the `TEST_FILE_DIR` variable the path to directory containing the test JPEG-LS compressed NetCDF file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "TEST_FILE_DIR = os.path.join(os.environ[\"EPCT_TEST_DATA_DIR\"], \"MTG\", \"MTGFCIL1\")\n",
    "COMP_FILE = os.path.join(\n",
    "    TEST_FILE_DIR,\n",
    "    \"W_XX-EUMETSAT-Darmstadt_IMG+SAT_MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY--\"\n",
    "    \"DIS-NC4E_C_EUMT_20200405120015_GTT_DEV_20200405115500_20200405115515_N_JLS_T_0072_0021.nc\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Usage with Python"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ``xarray``"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's import both the ``xarray`` and ``fcidecomp`` libraries, which loads the FCIDECOMP decompression filter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import xarray as xr\n",
    "\n",
    "import fcidecomp"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's open an JPEG-LS compressed file, loading a specific group of the netCDF file as an ``xarray`` dataset "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ds = xr.open_dataset(COMP_FILE, group='data/ir_105/measured')\n",
    "ds"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's plot the ``effective_radiace`` variable, showing that the file has been correctly uncompressed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ds.effective_radiance.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ``netCDF4``"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's import both the ``matplotlib``, ``netCDF4`` and ``fcidecomp`` libraries, which loads the FCIDECOMP decompression filter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import netCDF4 as nc\n",
    "\n",
    "import fcidecomp"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now open the JPEG-LS compressed netCDF file and load it into the ``ds`` variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ds = nc.Dataset(COMP_FILE, \"r\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's extract values of the band ``'data/ir_105/measured/effective_radiance`` and assign the resulting masked array to the ``band`` variable, and let's print its content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "band = ds['data/ir_105/measured/effective_radiance'][:]\n",
    "band"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Usage with netCDF-C based CLI tools"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ``nccopy``"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following line is actually run in the shell"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!nccopy -F none {COMP_FILE} {os.path.join(TEST_FILE_DIR, \"uncompressed_nccopy.nc\")}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's open the output of the previous shell command, which create a decompressed version of the JPEG-LS compressed test file ``COMP_FILE``"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ds = xr.open_dataset(os.path.join(TEST_FILE_DIR, \"uncompressed_nccopy.nc\"), group='data/ir_105/measured')\n",
    "ds"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's plot the ``effective_radiace`` variable, showing that the file has been correctly uncompressed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ds.effective_radiance.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Usage with the EUMETSAT Data Tailor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Decompression of JPEG-LS compressed netCDF files with the EUMETSAT Data Tailor software is enabled through its ``epct_plugin_mtg`` plugin."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, let's import the ``epct.api`` module and the ``xarray`` library"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import xarray as xr\n",
    "\n",
    "from epct import api"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's check that the plugin is correctly installed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "api.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's run the chain to decompress the JPEG-LS compressed netCDF file ``COMP_FILE``"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "chain_config = {\n",
    "    \"product\":   \"MTGFCIL1\", \n",
    "    \"format\": \"netcdf4_satellite\"\n",
    "}\n",
    "target_dir = './'\n",
    "product_paths = [COMP_FILE]\n",
    "output_products = api.run_chain(\n",
    "    product_paths, \n",
    "    chain_config=chain_config, \n",
    "    target_dir=target_dir, \n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's open the decompressed output's group ``data/ir_105/measured`` as an ``xarray`` dataset and plot its ``effective_radiance`` variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ds = xr.open_dataset(output_products[0], group='data/ir_105/measured')\n",
    "ds.effective_radiance.plot()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "fcidecomp_py39",
   "language": "python",
   "name": "fcidecomp_py39"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
