{ "cells": [ { "cell_type": "markdown", "id": "0d6fecdf-48c0-4745-b802-2117fb3137cf", "metadata": {}, "source": [ "# Introduction" ] }, { "cell_type": "markdown", "id": "15a05d43-0bf5-48d3-9c88-6074eed82a04", "metadata": {}, "source": [ "## Overview\n", "### In this tutorial, you learn:\n", "\n", "* Basics of Cupy-Xarray\n", "* Creating and handling Xarray DataArrays on GPUs\n", "* Data Transfer Between Host and Device\n", "\n", "## Prerequisites\n", "\n", "| Concepts | Importance | Notes |\n", "| --- | --- | --- |\n", "| [Familiarity with NumPy](https://foundations.projectpythia.org/core/numpy.html) | Necessary | |\n", "| [Basics of Cupy](Notebook0_Introduction) | Necessary | |\n", "| [Familiarity with Xarray](https://foundations.projectpythia.org/core/xarray.html) | Necessary | |\n", "\n", "- **Time to learn**: 10 minutes\n", "\n", "\n", "## Introduction " ] }, { "cell_type": "markdown", "id": "8cfcfa1f-f74a-405b-957d-3c925aca7eb4", "metadata": {}, "source": [ "Xarray is a powerful library for working with labeled multi-dimensional arrays in Python. It provides a convenient and intuitive way to manipulate large and complex datasets, and is built on top of NumPy. CuPy, on the other hand, is a library that allows for GPU-accelerated computing with Python and is compatible with NumPy.\n", "\n", "When used together, Xarray and CuPy can provide an easy way to take advantage of GPU acceleration for scientific computing tasks.\n", "\n", "Xarray can wrap custom duck array objects (i.e. NumPy-like arrays) that follow specific protocols. \n", "\n", "CuPy-Xarray provides an interface for using CuPy in Xarray, providing [accessors](https://docs.xarray.dev/en/stable/internals/extending-xarray.html) on the Xarray objects. \n", "\n", "This tutorial showcases the use of `cupy-xarray`, which offers a `cupy` accessor that allows access to cupy-specific features." ] }, { "cell_type": "markdown", "id": "77343efb-de6d-423c-b1cd-934c5d6d68e1", "metadata": {}, "source": [ "First, let's import our packages\n", "\n" ] }, { "cell_type": "code", "execution_count": 1, "id": "55c72b7d-8899-4e2f-9432-e9cf1531cbdf", "metadata": {}, "outputs": [], "source": [ "## Import NumPy and CuPy\n", "import cupy as cp\n", "import numpy as np\n", "import xarray as xr\n", "import cupy_xarray # Adds .cupy to Xarray objects" ] }, { "cell_type": "markdown", "id": "4ed42841-264a-4eb6-9f82-be9a463be816", "metadata": {}, "source": [ "### Creating Xarray DataArray with CuPy" ] }, { "cell_type": "markdown", "id": "573bb115-0e77-4f86-be9f-9ee6ac1c6f9b", "metadata": {}, "source": [ "In the previous tutorial, we learned how to create a NumPy and CuPy array:" ] }, { "cell_type": "code", "execution_count": 2, "id": "4b91fc01-9e99-4b01-b700-9b4802b7ef14", "metadata": {}, "outputs": [], "source": [ "arr_cpu = np.random.rand(10, 10, 10)\n", "arr_gpu = cp.random.rand(10, 10, 10)" ] }, { "cell_type": "markdown", "id": "2136402a-0224-4a2d-828f-99d571c4524b", "metadata": {}, "source": [ "We can create the Xarray DataArray using the CuPy array or NumPy array as the data source in a similar fashion:" ] }, { "cell_type": "code", "execution_count": 3, "id": "67e76867-373b-4011-b7c6-cb9b737e54c7", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
<xarray.DataArray (x: 10, y: 10, time: 10)>\n", "array([[[9.86345808e-01, 4.17660665e-01, 1.88327552e-01, 8.90280942e-01,\n", " 2.21689274e-01, 3.17943609e-01, 5.00880587e-01, 7.53337533e-01,\n", " 3.59031996e-01, 1.91030893e-01],\n", " [1.33144043e-02, 5.02596284e-01, 5.42913172e-01, 5.23846968e-01,\n", " 4.05313585e-01, 9.46118837e-01, 1.06548298e-01, 3.47524404e-01,\n", " 1.52843324e-01, 8.48253778e-01],\n", " [4.33649929e-01, 6.23728979e-01, 6.47445402e-01, 9.03623126e-01,\n", " 9.05012105e-01, 4.06989322e-03, 4.19896664e-01, 8.60406673e-02,\n", " 7.41788571e-01, 6.22665340e-01],\n", " [1.74267952e-01, 6.14897148e-01, 5.01242328e-01, 6.66759345e-01,\n", " 8.44182632e-01, 3.19620076e-01, 7.90701915e-01, 2.43897985e-01,\n", " 8.25956047e-01, 6.06534832e-01],\n", " [5.28111326e-01, 7.42343565e-01, 8.05094324e-02, 8.84691476e-01,\n", " 1.69856723e-02, 3.32512453e-01, 6.67738160e-01, 7.06905069e-01,\n", " 5.16369945e-01, 2.71965903e-01],\n", " [2.81638568e-01, 2.89389278e-01, 8.19006807e-01, 3.53878654e-01,\n", " 9.21084664e-02, 5.69411698e-01, 8.56797393e-01, 3.24107223e-01,\n", " 8.15087813e-01, 4.70350855e-01],\n", " [5.70124339e-01, 7.92088214e-01, 9.33540441e-01, 9.88027072e-01,\n", " 9.05585677e-01, 5.28417548e-01, 4.40169554e-01, 1.40924601e-01,\n", "...\n", " 5.29797743e-01, 9.87589722e-01, 9.18635655e-01, 8.68580278e-01,\n", " 4.71548324e-01, 3.64257635e-01],\n", " [6.42229124e-01, 2.33643023e-02, 5.85033551e-01, 8.80436137e-02,\n", " 7.07996956e-01, 4.40586920e-01, 3.10391741e-01, 1.22763638e-02,\n", " 8.02412664e-01, 4.33761051e-01],\n", " [1.24780820e-01, 3.53875474e-01, 8.36031716e-01, 2.84138174e-02,\n", " 3.57476794e-01, 2.44890794e-02, 1.47504786e-02, 3.19465404e-01,\n", " 2.91984584e-01, 3.39490525e-01],\n", " [2.04021642e-01, 4.71267303e-01, 9.03187717e-02, 3.83928128e-01,\n", " 5.96265409e-01, 3.17287239e-01, 3.22413673e-01, 8.38235070e-01,\n", " 9.58316554e-01, 9.73589612e-01],\n", " [6.13802208e-01, 8.70356525e-01, 5.17350919e-01, 7.72374828e-03,\n", " 5.35340510e-01, 8.89268388e-01, 6.93943330e-01, 6.29953006e-01,\n", " 1.70230716e-01, 7.16573680e-01],\n", " [8.44214598e-01, 3.35186917e-01, 8.78891352e-01, 1.98027834e-01,\n", " 6.36005433e-01, 1.21753118e-01, 6.48103717e-01, 8.68341345e-01,\n", " 7.81023406e-01, 4.45064620e-01],\n", " [3.85731750e-01, 8.02230895e-01, 6.41415045e-01, 7.60886886e-01,\n", " 2.00746550e-01, 3.76787007e-01, 6.68073723e-01, 7.87222270e-01,\n", " 6.75273015e-01, 8.63705777e-01]]])\n", "Dimensions without coordinates: x, y, time
<xarray.DataArray (x: 10, y: 10, time: 10)>\n", "array([[[2.88196731e-01, 3.71102840e-01, 8.22413516e-01, 7.61603373e-01,\n", " 2.14247694e-01, 6.08972260e-01, 6.35605124e-01, 4.51735394e-02,\n", " 3.56580833e-01, 3.33245593e-01],\n", " [9.56686233e-02, 3.09634487e-01, 5.72034429e-01, 8.64203361e-01,\n", " 5.44551902e-01, 4.54445926e-01, 1.21606888e-01, 2.30160410e-01,\n", " 6.14639953e-01, 7.73246535e-01],\n", " [8.03011705e-01, 2.69969912e-01, 2.03781951e-01, 6.64806547e-01,\n", " 4.93709552e-01, 2.60248353e-01, 6.82195033e-01, 6.75837492e-01,\n", " 5.07293067e-01, 6.45924343e-01],\n", " [1.03968071e-01, 1.31787260e-01, 2.31666523e-02, 2.90727455e-01,\n", " 6.22514068e-02, 9.54996781e-01, 1.38868633e-01, 3.18043546e-01,\n", " 9.94141764e-01, 6.52825114e-01],\n", " [6.72144360e-01, 9.25109790e-01, 9.24907616e-01, 9.97835547e-01,\n", " 1.30089788e-01, 3.28381980e-01, 9.47761645e-01, 2.15451004e-01,\n", " 1.55072912e-01, 2.84564825e-01],\n", " [5.32157180e-01, 4.05812774e-01, 6.65152077e-01, 1.62793186e-01,\n", " 8.38375837e-01, 4.38498164e-01, 3.93970103e-01, 3.25181026e-01,\n", " 8.43314943e-01, 6.37218468e-01],\n", " [9.47935236e-01, 1.39071514e-01, 3.34994498e-01, 7.42907508e-01,\n", " 1.13865457e-01, 3.69531071e-01, 6.58907523e-01, 4.10997683e-01,\n", "...\n", " 5.01101857e-01, 6.76530919e-01, 6.01550513e-01, 1.91761020e-01,\n", " 2.01591335e-01, 3.73443454e-01],\n", " [8.72935075e-01, 9.28175014e-01, 7.03819938e-01, 4.25757273e-01,\n", " 6.80355431e-01, 1.22351044e-01, 8.22086635e-03, 9.23118431e-01,\n", " 8.00040998e-02, 3.51963004e-01],\n", " [5.30917733e-01, 1.73025731e-03, 5.46551386e-01, 3.41904305e-01,\n", " 6.11276326e-01, 7.83903426e-01, 7.67650251e-01, 9.27383669e-02,\n", " 5.99146336e-01, 1.44674661e-02],\n", " [9.32478257e-02, 6.51279678e-01, 3.40032365e-01, 6.66761485e-02,\n", " 3.88243075e-01, 3.06181721e-02, 5.58666002e-01, 3.10356676e-01,\n", " 6.46523629e-01, 1.19013418e-01],\n", " [1.81940990e-01, 3.89650142e-01, 9.98204973e-01, 4.39178186e-02,\n", " 6.88137446e-02, 7.61541679e-02, 6.26075251e-01, 9.14708720e-01,\n", " 4.45414011e-01, 5.16678456e-01],\n", " [8.51618677e-01, 6.81900815e-01, 6.66821786e-01, 8.75685884e-01,\n", " 2.90499242e-01, 3.25977864e-01, 3.67627054e-01, 3.93770674e-01,\n", " 7.40898577e-01, 3.50451112e-02],\n", " [7.06374026e-01, 7.19519511e-01, 1.79160522e-01, 8.81425785e-01,\n", " 3.51431945e-01, 4.11507382e-01, 6.86088790e-01, 3.04671156e-01,\n", " 5.70729870e-01, 7.76584760e-01]]])\n", "Dimensions without coordinates: x, y, time
<xarray.DataArray (x: 10, y: 10, time: 10)>\n", "array([[[2.88196731e-01, 3.71102840e-01, 8.22413516e-01, 7.61603373e-01,\n", " 2.14247694e-01, 6.08972260e-01, 6.35605124e-01, 4.51735394e-02,\n", " 3.56580833e-01, 3.33245593e-01],\n", " [9.56686233e-02, 3.09634487e-01, 5.72034429e-01, 8.64203361e-01,\n", " 5.44551902e-01, 4.54445926e-01, 1.21606888e-01, 2.30160410e-01,\n", " 6.14639953e-01, 7.73246535e-01],\n", " [8.03011705e-01, 2.69969912e-01, 2.03781951e-01, 6.64806547e-01,\n", " 4.93709552e-01, 2.60248353e-01, 6.82195033e-01, 6.75837492e-01,\n", " 5.07293067e-01, 6.45924343e-01],\n", " [1.03968071e-01, 1.31787260e-01, 2.31666523e-02, 2.90727455e-01,\n", " 6.22514068e-02, 9.54996781e-01, 1.38868633e-01, 3.18043546e-01,\n", " 9.94141764e-01, 6.52825114e-01],\n", " [6.72144360e-01, 9.25109790e-01, 9.24907616e-01, 9.97835547e-01,\n", " 1.30089788e-01, 3.28381980e-01, 9.47761645e-01, 2.15451004e-01,\n", " 1.55072912e-01, 2.84564825e-01],\n", " [5.32157180e-01, 4.05812774e-01, 6.65152077e-01, 1.62793186e-01,\n", " 8.38375837e-01, 4.38498164e-01, 3.93970103e-01, 3.25181026e-01,\n", " 8.43314943e-01, 6.37218468e-01],\n", " [9.47935236e-01, 1.39071514e-01, 3.34994498e-01, 7.42907508e-01,\n", " 1.13865457e-01, 3.69531071e-01, 6.58907523e-01, 4.10997683e-01,\n", "...\n", " 5.01101857e-01, 6.76530919e-01, 6.01550513e-01, 1.91761020e-01,\n", " 2.01591335e-01, 3.73443454e-01],\n", " [8.72935075e-01, 9.28175014e-01, 7.03819938e-01, 4.25757273e-01,\n", " 6.80355431e-01, 1.22351044e-01, 8.22086635e-03, 9.23118431e-01,\n", " 8.00040998e-02, 3.51963004e-01],\n", " [5.30917733e-01, 1.73025731e-03, 5.46551386e-01, 3.41904305e-01,\n", " 6.11276326e-01, 7.83903426e-01, 7.67650251e-01, 9.27383669e-02,\n", " 5.99146336e-01, 1.44674661e-02],\n", " [9.32478257e-02, 6.51279678e-01, 3.40032365e-01, 6.66761485e-02,\n", " 3.88243075e-01, 3.06181721e-02, 5.58666002e-01, 3.10356676e-01,\n", " 6.46523629e-01, 1.19013418e-01],\n", " [1.81940990e-01, 3.89650142e-01, 9.98204973e-01, 4.39178186e-02,\n", " 6.88137446e-02, 7.61541679e-02, 6.26075251e-01, 9.14708720e-01,\n", " 4.45414011e-01, 5.16678456e-01],\n", " [8.51618677e-01, 6.81900815e-01, 6.66821786e-01, 8.75685884e-01,\n", " 2.90499242e-01, 3.25977864e-01, 3.67627054e-01, 3.93770674e-01,\n", " 7.40898577e-01, 3.50451112e-02],\n", " [7.06374026e-01, 7.19519511e-01, 1.79160522e-01, 8.81425785e-01,\n", " 3.51431945e-01, 4.11507382e-01, 6.86088790e-01, 3.04671156e-01,\n", " 5.70729870e-01, 7.76584760e-01]]])\n", "Dimensions without coordinates: x, y, time