MATLAB Answers

Downloading HYCOM data using Matlab and OPeNDAP

138 ビュー (過去 30 日間)
Amitava Guha
Amitava Guha 2020 年 6 月 5 日
編集済み: Gabriel Ruiz Martinez 2021 年 4 月 19 日 17:54
I am trying to download HYCOM (www.hycom.org) data using the OPeNDAP access method and MATLAB. The data url can be generated by selecting desired variables from this link. I need to download the data using scripts to automate the download of many years of data.
My MATLAB code looks like below, which you should be able to run as well without any modification:
% This script download the Global HYCOM reanalysis data for a given time step
for i=0:2863
timestep = 1; % This value need to be adjusted for duration of the record needed
j=i+timestep;
url = ['http://tds.hycom.org/thredds/dodsC/GLBv0.08/expt_53.X/data/1994?depth[0:1:39],lat[0:1:3250],lon[0:1:4499],time['...
num2str(i) ':1:' num2str(j) ...
'],tau[0:1:2864],water_u[0:1:0][0:1:0][0:1:0][0:1:0],water_u_bottom[0:1:0][0:1:0][0:1:0],water_v[0:1:0][0:1:0][0:1:0][0:1:0],water_v_bottom[0:1:0][0:1:0][0:1:0],water_temp[0:1:0][0:1:0][0:1:0][0:1:0],water_temp_bottom[0:1:0][0:1:0][0:1:0],salinity[0:1:0][0:1:0][0:1:0][0:1:0],salinity_bottom[0:1:0][0:1:0][0:1:0],surf_el[0:1:0][0:1:0][0:1:0]'];
info = ncinfo(url);
% Successfully read these variables
depth = ncread(url, 'depth');
lat = ncread(url, 'lat');
lon = ncread(url, 'lon');
time = ncread(url, 'time');
% Trouble reading below variables
water_u = ncread(url, 'water_u');
water_v = ncread(url, 'water_v');
water_u_bottom = ncread(url, 'water_u_bottom');
water_v_bottom = ncread(url, 'water_v_bottom');
surf_el = ncread(url, 'surf_el');
water_temp = ncread(url, 'water_temp');
end
The scripts reads the depth, lat, lon, time fine, but gives error in reading the water_u variable on wards. The error message is:
Error using netcdflib
The NetCDF library encountered an error during execution of 'getVarShort' function - 'Index exceeds dimension bound
(NC_EINVALCOORDS)'.
Error in netcdf.getVar (line 136)
data = netcdflib(funcstr,ncid,varid,varargin{:});
Error in internal.matlab.imagesci.nc/read (line 605)
data = netcdf.getVar(gid, varid);
Error in ncread (line 58)
vardata = ncObj.read(varName, varargin{:});
I am not sure what is wrong with my code, and couldn't find any answer searching online so far. Appreciate your help.
  1 件のコメント
Belinda Finlay
Belinda Finlay 2020 年 7 月 31 日
Did you solve this as I am having the same issue (although I got the surf_el values)?

サインインしてコメントする。

採用された回答

Amitava Guha
Amitava Guha 2021 年 1 月 4 日
I found out that downloading data from HYCOM is easier using Powershell script. Below is my code for a single grid point and for one year. I have been successful in running 5 such scripts at the same time, downloading 5 years of data for 1 grid point in 4-6 hour timeframe.
# This script is used to download current hindcast data from www.hycom.org
# Result directory
# --------------------------------------------------------
$resultDirectory = $PSScriptRoot + "\Data"
If (!(test-path $resultDirectory))
{
md $resultDirectory
}
# Input
# -------------------------------------------------------
# Select coordinate for region of interest
# Location: Latitude 6° 20' S , Longitude: 11° 15' E
#East
$east = 11.30
#West
$west = 11.25
#South
$south = -6.33
#North
$north = -6.31
$date_start = '01-Jan-1994 12:00:00'
$date_end = '31-Dec-1994 23:00:00'
# --------------------------------------------------------
# Converting dates to datetime
$startDate = [datetime]::ParseExact($date_start,'dd-MMM-yyyy HH:mm:ss',$null)
$endDate = [datetime]::ParseExact($date_end,'dd-MMM-yyyy HH:mm:ss',$null)
Write-Host "Downloading data from " $startDate.ToString('yyyy-MM-ddTHH:mm:ssZ') " to " $endDate.ToString('yyyy-MM-ddTHH:mm:ssZ')
for ($time = $startDate; $time -le $endDate; $time=$time.AddHours(3)){
$error_flag = 1
while ($error_flag -eq 1){
# Example url. Do not delete. Used for date time format reference purposes
#$url = "http://ncss.hycom.org/thredds/ncss/GLBv0.08/expt_53.X/data/2015?var=water_u&var=water_v&north=8.6&west=-57.3&east=-57.25&south=8.5&time=2015-01-01T18:00:00Z&accept=netcdf4"
# Download url
$url = "http://ncss.hycom.org/thredds/ncss/GLBv0.08/expt_53.X/data/" + $time.ToString('yyyy') + "?var=water_u&var=water_v&north=" + $north.ToString() + "&west=" + $west.ToString() + "&east=" + $east.ToString() + "&south=" + $south.ToString() + "&time=" + $time.ToString('yyyy-MM-ddTHH:mm:ssZ') + "&accept=netcdf4"
# Output file name
$fileName = $time.ToString('yyyyMMdd_HH') + ".nc"
$output = $PSScriptRoot + "\Data\" + $fileName
Try{
# Creating a web client which has the download file functionality
#WebProxy = New-Object System.Net.WebProxy("hoeprx01.na.xom.com:8080",$true)
$WebClient = New-Object System.Net.WebClient
#$WebClient.Proxy=$WebProxy
$WebClient.DownloadFile($url,$output)
Write-Host "Successfully downloaded file:" $fileName
$error_flag = 0
}
Catch {
Write-Host $_.Exception.Message`n
$error_flag = 1
Write-Host "Retrying downloading file:" $fileName " ...."
}
}
}
  1 件のコメント
Muhammad Nadzrin Nazri
Muhammad Nadzrin Nazri 2021 年 3 月 3 日
Dear Amitava,
Can you guide me to download using power shell? My area (south china sea):
North: 14
South: 0
East: 99
West: 117
I want 1 hourly, surface data from jan 2019 - Aug 2020:

サインインしてコメントする。

その他の回答 (2 件)

Zhiguo Mei
Zhiguo Mei 2020 年 11 月 26 日
Maybe you can try this "water_u = ncread(url, 'water_u',[1,1,1,1],[200,200,40,1]);"
Reduce the size of the variable you want read at once,and it always works for me

Gabriel Ruiz Martinez
Gabriel Ruiz Martinez 2021 年 4 月 19 日 1:54
編集済み: Gabriel Ruiz Martinez 2021 年 4 月 19 日 17:53
HI Amitava, thanks for sharing your Powershell script. Recently, I faced off the same problem as you. Using the NCTOOLBOX (Schilining et al., 2009), I did a Matlab script to download the HYCOM data from opeNDAP.
The code can find in Matlab File Exchange:
  2 件のコメント
Gabriel Ruiz Martinez
Gabriel Ruiz Martinez 2021 年 4 月 19 日 16:09
Hi Muhammad,
I recognize you have to download points from an entire region. In this case, you should run my function for every point (or node). You should change the dataUrl and dataUrl1 variables with the Hindcast HYCOM dataset URL that you need to use.
My advice is to write a script where you save in vectors the inputs that function requires and then... call the function for every point.
Reading the OpeNDAP Quickstart doc, if I understood well, you can download a maximum of 50 MB of data. I think if you want to download an entire region of data, the data will exceed 50 MB, sending an error message. For this reason, maybe you need to download node by node. Taking into account this fact, I wrote the function.
Notice the URL you shared is NetcdfSubset URL, the function HYCOMnc2txtS requires the OPENDAP URL:
(I would think this the URL you need, but I don't sure)
Regards.

サインインしてコメントする。

タグ

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by