Using CleF 2: advanced command line functionalities

Using CleF 2: advanced command line functionalities#

Paola Petrelli, CLEX CMS

In the first CleF blog we covered the basic command line options:

exploring the command and sub-commands options
passing constraints to query CMIP5, CORDEX and CMIP6 data
searching only the locally stored data or the ESGF published collection
how to check which data has been published but not yet downloaded locally
and finally how to request for data to be downloaded.

In the next blog we will see how to use CleF functions directly in your own python scripts. In this blog we are still focusing on the command line and in particular on how to:

save the query results as a csv file
get a summary of available data
run more complex queries
get extra information from the ESDOC documentation as errate and citations

A reminder that CleF is currently installed in the CMS conda module analysis3 and analysis3-unstable for the latest version. This is managed by the CMS and is available simply by running

module use /g/data3/hh5/public/modules
module load conda/analysis3

You need to be a member of hh5 to use the modules and of one of the CMIP projects: oi10,rr3, fs38, al33 to access the data and the clef database.

# run this if you haven't done so already in the terminal
#!module use /g/data3/hh5/public/modules
#!module load conda/analysis3-unstable

CSV file output#

The --csv option added to the command line will output the query results in a csv file. Rather than getting only the files path, it will list all the available attributes.
This currently works only with the --local and --remote option, it doesn’t yet work for the standard search, which is basically a combination of the two.

!clef --local cmip5 -v pr -v tas -e piControl  -en r1i1p1 -t Amon --csv

/g/data/al33/replicas/CMIP5/combined/BCC/bcc-csm1-1-m/piControl/mon/atmos/Amon/r1i1p1/v20120705/pr
/g/data/al33/replicas/CMIP5/combined/BCC/bcc-csm1-1/piControl/mon/atmos/Amon/r1i1p1/v1/pr
/g/data/al33/replicas/CMIP5/combined/BNU/BNU-ESM/piControl/mon/atmos/Amon/r1i1p1/v20120626/pr
/g/data/al33/replicas/CMIP5/combined/CCCma/CanESM2/piControl/mon/atmos/Amon/r1i1p1/v20120412/pr
/g/data/al33/replicas/CMIP5/combined/CCCma/CanESM2/piControl/mon/atmos/Amon/r1i1p1/v20120623/pr
/g/data/al33/replicas/CMIP5/combined/CMCC/CMCC-CESM/piControl/mon/atmos/Amon/r1i1p1/v20121023/pr
/g/data/al33/replicas/CMIP5/combined/CMCC/CMCC-CESM/piControl/mon/atmos/Amon/r1i1p1/v20170725/pr
/g/data/al33/replicas/CMIP5/combined/CMCC/CMCC-CM/piControl/mon/atmos/Amon/r1i1p1/v20121008/pr
/g/data/al33/replicas/CMIP5/combined/CMCC/CMCC-CM/piControl/mon/atmos/Amon/r1i1p1/v20170725/pr
/g/data/al33/replicas/CMIP5/combined/CMCC/CMCC-CMS/piControl/mon/atmos/Amon/r1i1p1/v20120717/pr
/g/data/al33/replicas/CMIP5/combined/CMCC/CMCC-CMS/piControl/mon/atmos/Amon/r1i1p1/v20170725/pr
/g/data/al33/replicas/CMIP5/combined/CNRM-CERFACS/CNRM-CM5/piControl/mon/atmos/Amon/r1i1p1/v20110701/pr
/g/data/al33/replicas/CMIP5/combined/FIO/FIO-ESM/piControl/mon/atmos/Amon/r1i1p1/v20120522/pr
/g/data/al33/replicas/CMIP5/combined/FIO/fio-esm/piControl/mon/atmos/Amon/r1i1p1/v20120522/pr
/g/data/al33/replicas/CMIP5/combined/ICHEC/EC-EARTH/piControl/mon/atmos/Amon/r1i1p1/v20120622/pr
/g/data/al33/replicas/CMIP5/combined/ICHEC/EC-EARTH/piControl/mon/atmos/Amon/r1i1p1/v20131120/pr
/g/data/al33/replicas/CMIP5/combined/INM/inmcm4/piControl/mon/atmos/Amon/r1i1p1/v20130207/pr
/g/data/al33/replicas/CMIP5/combined/IPSL/IPSL-CM5A-LR/piControl/mon/atmos/Amon/r1i1p1/v20130506/pr
/g/data/al33/replicas/CMIP5/combined/IPSL/IPSL-CM5A-MR/piControl/mon/atmos/Amon/r1i1p1/v20111119/pr
/g/data/al33/replicas/CMIP5/combined/IPSL/IPSL-CM5B-LR/piControl/mon/atmos/Amon/r1i1p1/v20120114/pr
/g/data/al33/replicas/CMIP5/combined/LASG-CESS/FGOALS-g2/piControl/mon/atmos/Amon/r1i1p1/v1/pr
/g/data/al33/replicas/CMIP5/combined/LASG-CESS/FGOALS-g2/piControl/mon/atmos/Amon/r1i1p1/v20161204/pr
/g/data/al33/replicas/CMIP5/combined/LASG-IAP/FGOALS-s2/piControl/mon/atmos/Amon/r1i1p1/v20161204/pr
/g/data/al33/replicas/CMIP5/combined/LASG-IAP/FGOALS-s2/piControl/mon/atmos/Amon/r1i1p1/v4/pr
/g/data/al33/replicas/CMIP5/combined/MIROC/MIROC-ESM-CHEM/piControl/mon/atmos/Amon/r1i1p1/v20120710/pr
/g/data/al33/replicas/CMIP5/combined/MIROC/MIROC-ESM/piControl/mon/atmos/Amon/r1i1p1/v20120710/pr
/g/data/al33/replicas/CMIP5/combined/MIROC/MIROC4h/piControl/mon/atmos/Amon/r1i1p1/v20120628/pr
/g/data/al33/replicas/CMIP5/combined/MIROC/MIROC5/piControl/mon/atmos/Amon/r1i1p1/v20161012/pr
/g/data/al33/replicas/CMIP5/combined/MOHC/HadGEM2-CC/piControl/mon/atmos/Amon/r1i1p1/v20111109/pr
/g/data/al33/replicas/CMIP5/combined/MOHC/HadGEM2-ES/piControl/mon/atmos/Amon/r1i1p1/v20110524/pr
/g/data/al33/replicas/CMIP5/combined/MPI-M/MPI-ESM-LR/piControl/mon/atmos/Amon/r1i1p1/v20120602/pr
/g/data/al33/replicas/CMIP5/combined/MPI-M/MPI-ESM-MR/piControl/mon/atmos/Amon/r1i1p1/v20120602/pr
/g/data/al33/replicas/CMIP5/combined/MPI-M/MPI-ESM-P/piControl/mon/atmos/Amon/r1i1p1/v20120602/pr
/g/data/al33/replicas/CMIP5/combined/MRI/MRI-CGCM3/piControl/mon/atmos/Amon/r1i1p1/v20120701/pr
/g/data/al33/replicas/CMIP5/combined/NASA-GISS/GISS-E2-H-CC/piControl/mon/atmos/Amon/r1i1p1/v20160511/pr
/g/data/al33/replicas/CMIP5/combined/NASA-GISS/GISS-E2-H/piControl/mon/atmos/Amon/r1i1p1/v20160511/pr
/g/data/al33/replicas/CMIP5/combined/NASA-GISS/GISS-E2-R-CC/piControl/mon/atmos/Amon/r1i1p1/v20160802/pr
/g/data/al33/replicas/CMIP5/combined/NASA-GISS/GISS-E2-R/piControl/mon/atmos/Amon/r1i1p1/v20161004/pr
/g/data/al33/replicas/CMIP5/combined/NCAR/CCSM4/piControl/mon/atmos/Amon/r1i1p1/v20130325/pr
/g/data/al33/replicas/CMIP5/combined/NCAR/CCSM4/piControl/mon/atmos/Amon/r1i1p1/v20130510/pr
/g/data/al33/replicas/CMIP5/combined/NCAR/CCSM4/piControl/mon/atmos/Amon/r1i1p1/v20160829/pr
/g/data/al33/replicas/CMIP5/combined/NCC/NorESM1-M/piControl/mon/atmos/Amon/r1i1p1/v20110901/pr
/g/data/al33/replicas/CMIP5/combined/NCC/NorESM1-M/piControl/mon/atmos/Amon/r1i1p1/v20120412/pr
/g/data/al33/replicas/CMIP5/combined/NCC/NorESM1-M/piControl/mon/atmos/Amon/r1i1p1/v20200617/pr
/g/data/al33/replicas/CMIP5/combined/NCC/NorESM1-ME/piControl/mon/atmos/Amon/r1i1p1/v20120225/pr
/g/data/al33/replicas/CMIP5/combined/NCC/NorESM1-ME/piControl/mon/atmos/Amon/r1i1p1/v20130926/pr
/g/data/al33/replicas/CMIP5/combined/NCC/NorESM1-ME/piControl/mon/atmos/Amon/r1i1p1/v20200617/pr
/g/data/al33/replicas/CMIP5/combined/NIMR-KMA/HadGEM2-AO/piControl/mon/atmos/Amon/r1i1p1/v20130815/pr
/g/data/al33/replicas/CMIP5/combined/NOAA-GFDL/GFDL-CM3/piControl/mon/atmos/Amon/r1i1p1/v20120227/pr
/g/data/al33/replicas/CMIP5/combined/NOAA-GFDL/GFDL-ESM2G/piControl/mon/atmos/Amon/r1i1p1/v20120830/pr
/g/data/al33/replicas/CMIP5/combined/NOAA-GFDL/GFDL-ESM2M/piControl/mon/atmos/Amon/r1i1p1/v20130214/pr
/g/data/al33/replicas/CMIP5/combined/NSF-DOE-NCAR/CESM1-BGC/piControl/mon/atmos/Amon/r1i1p1/v20130216/pr
/g/data/al33/replicas/CMIP5/combined/NSF-DOE-NCAR/CESM1-CAM5/piControl/mon/atmos/Amon/r1i1p1/v20130313/pr
/g/data/al33/replicas/CMIP5/combined/NSF-DOE-NCAR/CESM1-FASTCHEM/piControl/mon/atmos/Amon/r1i1p1/v20121029/pr
/g/data/al33/replicas/CMIP5/combined/NSF-DOE-NCAR/CESM1-WACCM/piControl/mon/atmos/Amon/r1i1p1/v20130325/pr
/g/data/rr3/publications/CMIP5/output1/CSIRO-BOM/ACCESS1-0/piControl/mon/atmos/Amon/r1i1p1/latest/pr
/g/data/rr3/publications/CMIP5/output1/CSIRO-BOM/ACCESS1-3/piControl/mon/atmos/Amon/r1i1p1/latest/pr
/g/data/rr3/publications/CMIP5/output1/CSIRO-QCCCE/CSIRO-Mk3-6-0/piControl/mon/atmos/Amon/r1i1p1/files/pr_20110518
/g/data/rr3/publications/PMIP3/output/UNSW/CSIRO-Mk3L-1-2/piControl/mon/atmos/Amon/r1i1p1/files/pr_20170728
/g/data/rr3/publications/PMIP3/output/UNSW/CSIRO-Mk3L-1-2/piControl/mon/atmos/Amon/r1i1p1/v20170728/pr
/g/data/al33/replicas/CMIP5/combined/BCC/bcc-csm1-1-m/piControl/mon/atmos/Amon/r1i1p1/v20120705/tas
/g/data/al33/replicas/CMIP5/combined/BCC/bcc-csm1-1/piControl/mon/atmos/Amon/r1i1p1/v1/tas
/g/data/al33/replicas/CMIP5/combined/BNU/BNU-ESM/piControl/mon/atmos/Amon/r1i1p1/v20120626/tas
/g/data/al33/replicas/CMIP5/combined/CCCma/CanESM2/piControl/mon/atmos/Amon/r1i1p1/v20120412/tas
/g/data/al33/replicas/CMIP5/combined/CCCma/CanESM2/piControl/mon/atmos/Amon/r1i1p1/v20120623/tas
/g/data/al33/replicas/CMIP5/combined/CMCC/CMCC-CESM/piControl/mon/atmos/Amon/r1i1p1/v20121023/tas
/g/data/al33/replicas/CMIP5/combined/CMCC/CMCC-CESM/piControl/mon/atmos/Amon/r1i1p1/v20170725/tas
/g/data/al33/replicas/CMIP5/combined/CMCC/CMCC-CM/piControl/mon/atmos/Amon/r1i1p1/v20121008/tas
/g/data/al33/replicas/CMIP5/combined/CMCC/CMCC-CM/piControl/mon/atmos/Amon/r1i1p1/v20170725/tas
/g/data/al33/replicas/CMIP5/combined/CMCC/CMCC-CMS/piControl/mon/atmos/Amon/r1i1p1/v20120717/tas
/g/data/al33/replicas/CMIP5/combined/CMCC/CMCC-CMS/piControl/mon/atmos/Amon/r1i1p1/v20170725/tas
/g/data/al33/replicas/CMIP5/combined/CNRM-CERFACS/CNRM-CM5-2/piControl/mon/atmos/Amon/r1i1p1/v20130402/tas
/g/data/al33/replicas/CMIP5/combined/CNRM-CERFACS/CNRM-CM5/piControl/mon/atmos/Amon/r1i1p1/v20110701/tas
/g/data/al33/replicas/CMIP5/combined/FIO/FIO-ESM/piControl/mon/atmos/Amon/r1i1p1/v20120522/tas
/g/data/al33/replicas/CMIP5/combined/FIO/fio-esm/piControl/mon/atmos/Amon/r1i1p1/v20120522/tas
/g/data/al33/replicas/CMIP5/combined/ICHEC/EC-EARTH/piControl/mon/atmos/Amon/r1i1p1/v20120622/tas
/g/data/al33/replicas/CMIP5/combined/ICHEC/EC-EARTH/piControl/mon/atmos/Amon/r1i1p1/v20131120/tas
/g/data/al33/replicas/CMIP5/combined/INM/inmcm4/piControl/mon/atmos/Amon/r1i1p1/v20130207/tas
/g/data/al33/replicas/CMIP5/combined/IPSL/IPSL-CM5A-LR/piControl/mon/atmos/Amon/r1i1p1/v20130506/tas
/g/data/al33/replicas/CMIP5/combined/IPSL/IPSL-CM5A-MR/piControl/mon/atmos/Amon/r1i1p1/v20111119/tas
/g/data/al33/replicas/CMIP5/combined/IPSL/IPSL-CM5B-LR/piControl/mon/atmos/Amon/r1i1p1/v20120114/tas
/g/data/al33/replicas/CMIP5/combined/LASG-CESS/FGOALS-g2/piControl/mon/atmos/Amon/r1i1p1/v1/tas
/g/data/al33/replicas/CMIP5/combined/LASG-CESS/FGOALS-g2/piControl/mon/atmos/Amon/r1i1p1/v20161204/tas
/g/data/al33/replicas/CMIP5/combined/LASG-IAP/FGOALS-s2/piControl/mon/atmos/Amon/r1i1p1/v20161204/tas
/g/data/al33/replicas/CMIP5/combined/LASG-IAP/FGOALS-s2/piControl/mon/atmos/Amon/r1i1p1/v4/tas
/g/data/al33/replicas/CMIP5/combined/MIROC/MIROC-ESM-CHEM/piControl/mon/atmos/Amon/r1i1p1/v20120710/tas
/g/data/al33/replicas/CMIP5/combined/MIROC/MIROC-ESM/piControl/mon/atmos/Amon/r1i1p1/v20120710/tas
/g/data/al33/replicas/CMIP5/combined/MIROC/MIROC4h/piControl/mon/atmos/Amon/r1i1p1/v20120628/tas
/g/data/al33/replicas/CMIP5/combined/MIROC/MIROC5/piControl/mon/atmos/Amon/r1i1p1/v20161012/tas
/g/data/al33/replicas/CMIP5/combined/MOHC/HadGEM2-CC/piControl/mon/atmos/Amon/r1i1p1/v20111109/tas
/g/data/al33/replicas/CMIP5/combined/MOHC/HadGEM2-ES/piControl/mon/atmos/Amon/r1i1p1/v20110524/tas
/g/data/al33/replicas/CMIP5/combined/MPI-M/MPI-ESM-LR/piControl/mon/atmos/Amon/r1i1p1/v20120602/tas
/g/data/al33/replicas/CMIP5/combined/MPI-M/MPI-ESM-MR/piControl/mon/atmos/Amon/r1i1p1/v20120602/tas
/g/data/al33/replicas/CMIP5/combined/MPI-M/MPI-ESM-P/piControl/mon/atmos/Amon/r1i1p1/v20120602/tas
/g/data/al33/replicas/CMIP5/combined/MRI/MRI-CGCM3/piControl/mon/atmos/Amon/r1i1p1/v20120701/tas
/g/data/al33/replicas/CMIP5/combined/NASA-GISS/GISS-E2-H-CC/piControl/mon/atmos/Amon/r1i1p1/v20160511/tas
/g/data/al33/replicas/CMIP5/combined/NASA-GISS/GISS-E2-H/piControl/mon/atmos/Amon/r1i1p1/v20160511/tas
/g/data/al33/replicas/CMIP5/combined/NASA-GISS/GISS-E2-R-CC/piControl/mon/atmos/Amon/r1i1p1/v20160802/tas
/g/data/al33/replicas/CMIP5/combined/NASA-GISS/GISS-E2-R/piControl/mon/atmos/Amon/r1i1p1/v20161004/tas
/g/data/al33/replicas/CMIP5/combined/NCAR/CCSM4/piControl/mon/atmos/Amon/r1i1p1/v20130325/tas
/g/data/al33/replicas/CMIP5/combined/NCAR/CCSM4/piControl/mon/atmos/Amon/r1i1p1/v20130510/tas
/g/data/al33/replicas/CMIP5/combined/NCAR/CCSM4/piControl/mon/atmos/Amon/r1i1p1/v20160829/tas
/g/data/al33/replicas/CMIP5/combined/NCC/NorESM1-M/piControl/mon/atmos/Amon/r1i1p1/v20110901/tas
/g/data/al33/replicas/CMIP5/combined/NCC/NorESM1-M/piControl/mon/atmos/Amon/r1i1p1/v20120412/tas
/g/data/al33/replicas/CMIP5/combined/NCC/NorESM1-M/piControl/mon/atmos/Amon/r1i1p1/v20200617/tas
/g/data/al33/replicas/CMIP5/combined/NCC/NorESM1-ME/piControl/mon/atmos/Amon/r1i1p1/v20120225/tas
/g/data/al33/replicas/CMIP5/combined/NCC/NorESM1-ME/piControl/mon/atmos/Amon/r1i1p1/v20130926/tas
/g/data/al33/replicas/CMIP5/combined/NCC/NorESM1-ME/piControl/mon/atmos/Amon/r1i1p1/v20200617/tas
/g/data/al33/replicas/CMIP5/combined/NIMR-KMA/HadGEM2-AO/piControl/mon/atmos/Amon/r1i1p1/v20130815/tas
/g/data/al33/replicas/CMIP5/combined/NOAA-GFDL/GFDL-CM3/piControl/mon/atmos/Amon/r1i1p1/v20120227/tas
/g/data/al33/replicas/CMIP5/combined/NOAA-GFDL/GFDL-ESM2G/piControl/mon/atmos/Amon/r1i1p1/v20120830/tas
/g/data/al33/replicas/CMIP5/combined/NOAA-GFDL/GFDL-ESM2M/piControl/mon/atmos/Amon/r1i1p1/v20130214/tas
/g/data/al33/replicas/CMIP5/combined/NSF-DOE-NCAR/CESM1-BGC/piControl/mon/atmos/Amon/r1i1p1/v20130216/tas
/g/data/al33/replicas/CMIP5/combined/NSF-DOE-NCAR/CESM1-CAM5/piControl/mon/atmos/Amon/r1i1p1/v20130313/tas
/g/data/al33/replicas/CMIP5/combined/NSF-DOE-NCAR/CESM1-FASTCHEM/piControl/mon/atmos/Amon/r1i1p1/v20121029/tas
/g/data/al33/replicas/CMIP5/combined/NSF-DOE-NCAR/CESM1-WACCM/piControl/mon/atmos/Amon/r1i1p1/v20130325/tas
/g/data/rr3/publications/CMIP5/output1/CSIRO-BOM/ACCESS1-0/piControl/mon/atmos/Amon/r1i1p1/latest/tas
/g/data/rr3/publications/CMIP5/output1/CSIRO-BOM/ACCESS1-3/piControl/mon/atmos/Amon/r1i1p1/latest/tas
/g/data/rr3/publications/CMIP5/output1/CSIRO-QCCCE/CSIRO-Mk3-6-0/piControl/mon/atmos/Amon/r1i1p1/files/tas_20110518
/g/data/rr3/publications/PMIP3/output/UNSW/CSIRO-Mk3L-1-2/piControl/mon/atmos/Amon/r1i1p1/files/tas_20170728
/g/data/rr3/publications/PMIP3/output/UNSW/CSIRO-Mk3L-1-2/piControl/mon/atmos/Amon/r1i1p1/v20170728/tas

!head -n 4 CMIP5_query.csv

,model,experiment,frequency,ensemble,cmor_table,version,variable,path,fdate,tdate,time_complete
0,BCC-CSM1.1(m),piControl,mon,r1i1p1,Amon,20120705,pr,/g/data/al33/replicas/CMIP5/combined/BCC/bcc-csm1-1-m/piControl/mon/atmos/Amon/r1i1p1/v20120705/pr,101,40012,
1,BCC-CSM1.1,piControl,mon,r1i1p1,Amon,1,pr,/g/data/al33/replicas/CMIP5/combined/BCC/bcc-csm1-1/piControl/mon/atmos/Amon/r1i1p1/v1/pr,101,50012,
2,BNU-ESM,piControl,mon,r1i1p1,Amon,20120626,pr,/g/data/al33/replicas/CMIP5/combined/BNU/BNU-ESM/piControl/mon/atmos/Amon/r1i1p1/v20120626/pr,14500101,20081231,True

In the first example I am passing --csv to a CMIP5 query with the --local option set.
I am hiding the cell output but the tool still prints the results on the display as well as creating the csv file.
The following example demonstrate how --csv works also for remote CMIP6 queries and with the flag --and. The --and flag allows for more complex filtering of the data and later we will see how it works.

!clef --remote cmip6 -v pr -v mrso -e piControl  -mi r1i1p1f1 --frequency mon --and variable_id --csv

ACCESS-CM2 / r1i1p1f1 versions: v20191112
ACCESS-ESM1-5 / r1i1p1f1 versions: v20191214
AWI-ESM-1-1-LR / r1i1p1f1 versions: v20200212
BCC-CSM2-MR / r1i1p1f1 versions: v20181012, v20181016
BCC-ESM1 / r1i1p1f1 versions: v20181211, v20181214
CAMS-CSM1-0 / r1i1p1f1 versions: v20190729, v20191212
CESM2 / r1i1p1f1 versions: v20190320
CESM2-FV2 / r1i1p1f1 versions: v20191120
CESM2-WACCM / r1i1p1f1 versions: v20190320
CESM2-WACCM-FV2 / r1i1p1f1 versions: v20191120
CIESM / r1i1p1f1 versions: v20200417
CMCC-CM2-SR5 / r1i1p1f1 versions: v20200616
CanESM5 / r1i1p1f1 versions: v20190429
E3SM-1-0 / r1i1p1f1 versions: v20190807, v20190719
E3SM-1-1 / r1i1p1f1 versions: v20191203, v20191029, v20200324
E3SM-1-1-ECA / r1i1p1f1 versions: v20191218, v20191216
EC-Earth3 / r1i1p1f1 versions: v20200312
EC-Earth3-Veg / r1i1p1f1 versions: v20200226
GISS-E2-1-G / r1i1p1f1 versions: v20180824
GISS-E2-1-H / r1i1p1f1 versions: v20190410
GISS-E2-2-G / r1i1p1f1 versions: v20191120
HadGEM3-GC31-LL / r1i1p1f1 versions: v20190628
HadGEM3-GC31-MM / r1i1p1f1 versions: v20190920
INM-CM4-8 / r1i1p1f1 versions: v20190605
INM-CM5-0 / r1i1p1f1 versions: v20190619
IPSL-CM6A-LR / r1i1p1f1 versions: v20200326
MCM-UA-1-0 / r1i1p1f1 versions: v20190731, v20191017
MIROC6 / r1i1p1f1 versions: v20190311, v20181212
MPI-ESM-1-2-HAM / r1i1p1f1 versions: v20190627
MPI-ESM1-2-HR / r1i1p1f1 versions: v20190710
MPI-ESM1-2-LR / r1i1p1f1 versions: v20190710
MRI-ESM2-0 / r1i1p1f1 versions: v20190222, v20190603
NorCPM1 / r1i1p1f1 versions: v20190914
NorESM2-LM / r1i1p1f1 versions: v20200217, v20190920
NorESM2-MM / r1i1p1f1 versions: v20191108
SAM0-UNICON / r1i1p1f1 versions: v20190910
TaiESM1 / r1i1p1f1 versions: v20200211, v20200302

!head -n 4 CMIP6_query.csv

,index,version,activity_id,dataset_id,experiment_id,frequency,grid,grid_label,member_id,nominal_resolution,source_id,source_type,sub_experiment_id,table_id,variable_id,score,comb
0,0,"('v20200211',)",CMIP,CMIP6.CMIP.AS-RCEC.TaiESM1.piControl.r1i1p1f1.Amon.pr.gn.v20200211|esgf.rcec.sinica.edu.tw,piControl,mon,finite-volume grid with 0.9x1.25 degree lat/lon resolution,gn,r1i1p1f1,100 km,TaiESM1,AOGCM,none,Amon,pr,1.0,"('pr',)"
1,1,"('v20200211',)",CMIP,CMIP6.CMIP.AS-RCEC.TaiESM1.piControl.r1i1p1f1.Amon.pr.gn.v20200211|esgf.rcec.sinica.edu.tw,piControl,mon,finite-volume grid with 0.9x1.25 degree lat/lon resolution,gn,r1i1p1f1,100 km,TaiESM1,AOGCM,none,Amon,pr,1.0,"('pr',)"
2,2,"('v20200211',)",CMIP,CMIP6.CMIP.AS-RCEC.TaiESM1.piControl.r1i1p1f1.Amon.pr.gn.v20200211|esgf.rcec.sinica.edu.tw,piControl,mon,finite-volume grid with 0.9x1.25 degree lat/lon resolution,gn,r1i1p1f1,100 km,TaiESM1,AOGCM,none,Amon,pr,1.0,"('pr',)"

Query summary option#

The --stats option added to the command line will print a summary of the query results
It works for both --local and --remote options, but not with the default query.
Currently it prints the following:

total number of models, followed by their names
total number of unique model-ensembles/members combinations
number of models that have N ensembles/members, followed by their names

!clef --local cmip5 -v pr -e rcp85 -t Amon --stats

Query summary

42 model/s are available:
ACCESS1.0 ACCESS1.3 BCC-CSM1.1 BCC-CSM1.1(m) BNU-ESM CCSM4 CESM1(BGC) CESM1(CAM5) CESM1(WACCM) CMCC-CESM CMCC-CM CMCC-CMS CNRM-CM5 CSIRO-Mk3.6.0 CanESM2 EC-EARTH FGOALS-s2 FGOALS_g2 FIO-ESM GFDL-CM3 GFDL-ESM2G GFDL-ESM2M GISS-E2-H GISS-E2-H-CC GISS-E2-R GISS-E2-R-CC HadGEM2-AO HadGEM2-CC HadGEM2-ES IPSL-CM5A-LR IPSL-CM5A-MR IPSL-CM5B-LR MIROC-ESM MIROC-ESM-CHEM MIROC5 MPI-ESM-LR MPI-ESM-MR MRI-CGCM3 MRI-ESM1 NorESM1-M NorESM1-ME inmcm4 

A total of 100 unique model-member combinations are available.

  26 model/s have 1 member/s:

     ACCESS1.0: r1i1p1
     ACCESS1.3: r1i1p1
     BCC-CSM1.1: r1i1p1
     BCC-CSM1.1(m): r1i1p1
     BNU-ESM: r1i1p1
     CESM1(BGC): r1i1p1
     CMCC-CESM: r1i1p1
     CMCC-CM: r1i1p1
     CMCC-CMS: r1i1p1
     FGOALS_g2: r1i1p1
     GFDL-CM3: r1i1p1
     GFDL-ESM2G: r1i1p1
     GFDL-ESM2M: r1i1p1
     GISS-E2-H-CC: r1i1p1
     GISS-E2-R-CC: r1i1p1
     HadGEM2-AO: r1i1p1
     IPSL-CM5A-MR: r1i1p1
     IPSL-CM5B-LR: r1i1p1
     MIROC-ESM: r1i1p1
     MIROC-ESM-CHEM: r1i1p1
     MPI-ESM-MR: r1i1p1
     MRI-CGCM3: r1i1p1
     MRI-ESM1: r1i1p1
     NorESM1-M: r1i1p1
     NorESM1-ME: r1i1p1
     inmcm4: r1i1p1

  7 model/s have 3 member/s:

     CESM1(CAM5): r1i1p1, r2i1p1, r3i1p1
     CESM1(WACCM): r2i1p1, r3i1p1, r4i1p1
     FGOALS-s2: r1i1p1, r2i1p1, r3i1p1
     FIO-ESM: r1i1p1, r2i1p1, r3i1p1
     HadGEM2-CC: r1i1p1, r2i1p1, r3i1p1
     MIROC5: r1i1p1, r2i1p1, r3i1p1
     MPI-ESM-LR: r1i1p1, r2i1p1, r3i1p1

  2 model/s have 4 member/s:

     HadGEM2-ES: r1i1p1, r2i1p1, r3i1p1, r4i1p1
     IPSL-CM5A-LR: r1i1p1, r2i1p1, r3i1p1, r4i1p1

  4 model/s have 5 member/s:

     CNRM-CM5: r10i1p1, r1i1p1, r2i1p1, r4i1p1, r6i1p1
     CanESM2: r1i1p1, r2i1p1, r3i1p1, r4i1p1, r5i1p1
     GISS-E2-H: r1i1p1, r1i1p2, r1i1p3, r2i1p1, r2i1p3
     GISS-E2-R: r1i1p1, r1i1p2, r1i1p3, r2i1p1, r2i1p3

  1 model/s have 6 member/s:

     CCSM4: r1i1p1, r2i1p1, r3i1p1, r4i1p1, r5i1p1, r6i1p1

  1 model/s have 9 member/s:

     EC-EARTH: r11i1p1, r12i1p1, r13i1p1, r1i1p1, r2i1p1, r6i1p1, r7i1p1, r8i1p1, r9i1p1

  1 model/s have 10 member/s:

     CSIRO-Mk3.6.0: r10i1p1, r1i1p1, r2i1p1, r3i1p1, r4i1p1, r5i1p1, r6i1p1, r7i1p1, r8i1p1, r9i1p1

!clef --local cmip5 -v cli -e rcp85 -e rcp45 -e rcp26 -t Amon --and experiment --stats

BNU-ESM / r1i1p1 versions: 20120510
CSIRO-Mk3.6.0 / r10i1p1 versions: v20110518
CSIRO-Mk3.6.0 / r1i1p1 versions: v20110518
CSIRO-Mk3.6.0 / r2i1p1 versions: v20110518
CSIRO-Mk3.6.0 / r3i1p1 versions: v20110518
CSIRO-Mk3.6.0 / r4i1p1 versions: v20110518
CSIRO-Mk3.6.0 / r5i1p1 versions: v20110518
CSIRO-Mk3.6.0 / r6i1p1 versions: v20110518
CSIRO-Mk3.6.0 / r7i1p1 versions: v20110518
CSIRO-Mk3.6.0 / r8i1p1 versions: v20110518
CSIRO-Mk3.6.0 / r9i1p1 versions: v20110518
CanESM2 / r1i1p1 versions: 20120718
FGOALS_g2 / r1i1p1 versions: 20161204
GFDL-CM3 / r1i1p1 versions: 20120227
GFDL-ESM2G / r1i1p1 versions: 20120412
GISS-E2-H / r1i1p1 versions: 20160512
GISS-E2-H / r1i1p2 versions: 20160512
GISS-E2-H / r1i1p3 versions: 20160512
GISS-E2-R / r1i1p1 versions: 20160803, 20160513, 20160512
GISS-E2-R / r1i1p2 versions: 20160513, 20160512
GISS-E2-R / r1i1p3 versions: 20160823, 20160513, 20160512
HadGEM2-ES / r1i1p1 versions: 20111128, 20111206, 20130430
HadGEM2-ES / r2i1p1 versions: 20111205, 20111129, 20120114
HadGEM2-ES / r3i1p1 versions: 20111208, 20111205, 20120114
HadGEM2-ES / r4i1p1 versions: 20111209, 20111205, 20120114
IPSL-CM5A-LR / r1i1p1 versions: 20110914, 20111103, 20120114
MIROC-ESM / r1i1p1 versions: 20120710
MIROC-ESM-CHEM / r1i1p1 versions: 20120710
MIROC5 / r1i1p1 versions: 20120710, 20161012
MIROC5 / r2i1p1 versions: 20120710
MIROC5 / r3i1p1 versions: 20120710
MPI-ESM-LR / r1i1p1 versions: 20120315
MPI-ESM-LR / r2i1p1 versions: 20120315
MPI-ESM-LR / r3i1p1 versions: 20120315
MPI-ESM-MR / r1i1p1 versions: 20120503
MRI-CGCM3 / r1i1p1 versions: 20120701
NorESM1-M / r1i1p1 versions: 20120412

Query summary

17 model/s are available:
BNU-ESM CSIRO-Mk3.6.0 CanESM2 FGOALS_g2 GFDL-CM3 GFDL-ESM2G GISS-E2-H GISS-E2-R HadGEM2-ES IPSL-CM5A-LR MIROC-ESM MIROC-ESM-CHEM MIROC5 MPI-ESM-LR MPI-ESM-MR MRI-CGCM3 NorESM1-M 

A total of 37 unique model-member combinations are available.

  11 model/s have 1 member/s:

     BNU-ESM: r1i1p1
     CanESM2: r1i1p1
     FGOALS_g2: r1i1p1
     GFDL-CM3: r1i1p1
     GFDL-ESM2G: r1i1p1
     IPSL-CM5A-LR: r1i1p1
     MIROC-ESM: r1i1p1
     MIROC-ESM-CHEM: r1i1p1
     MPI-ESM-MR: r1i1p1
     MRI-CGCM3: r1i1p1
     NorESM1-M: r1i1p1

  4 model/s have 3 member/s:

     GISS-E2-H: r1i1p1, r1i1p2, r1i1p3
     GISS-E2-R: r1i1p1, r1i1p2, r1i1p3
     MIROC5: r1i1p1, r2i1p1, r3i1p1
     MPI-ESM-LR: r3i1p1, r1i1p1, r2i1p1

  1 model/s have 4 member/s:

     HadGEM2-ES: r2i1p1, r3i1p1, r4i1p1, r1i1p1

  1 model/s have 10 member/s:

     CSIRO-Mk3.6.0: r8i1p1, r10i1p1, r1i1p1, r2i1p1, r3i1p1, r4i1p1, r5i1p1, r6i1p1, r7i1p1, r9i1p1

AND Filter#

We added the --and filter flag to perform more complex queries. As the name implies this option applies the AND operator to the query results. It is easier to explain by looking at an example.

Let’s say that you want to find all the CMIP6 models that have both daily precipitation (pr) and soil moisture (mrso) for a particular experiment (piControl). A way of doing it would be to select separately both variables and then work out which models had both.
With the --and option you can query for both variables at the same time and specify you want only the models, ensemble pairs that have both of them.

!clef --local cmip6 -v pr -v mrso -e piControl  -mi r1i1p1f1 --frequency mon --and variable_id

ACCESS-CM2 / r1i1p1f1 versions: v20191112
ACCESS-ESM1-5 / r1i1p1f1 versions: v20191214
AWI-ESM-1-1-LR / r1i1p1f1 versions: v20200212
BCC-CSM2-MR / r1i1p1f1 versions: v20181016, v20181012
BCC-ESM1 / r1i1p1f1 versions: v20181214, v20181211
CAMS-CSM1-0 / r1i1p1f1 versions: v20191212, v20190729
CESM2 / r1i1p1f1 versions: v20190320
CESM2-FV2 / r1i1p1f1 versions: v20191120
CESM2-WACCM / r1i1p1f1 versions: v20190320
CESM2-WACCM-FV2 / r1i1p1f1 versions: v20191120
CIESM / r1i1p1f1 versions: v20200417
CMCC-CM2-SR5 / r1i1p1f1 versions: v20200616
CanESM5 / r1i1p1f1 versions: v20190429
E3SM-1-0 / r1i1p1f1 versions: v20190807, v20190719
E3SM-1-1 / r1i1p1f1 versions: v20200324, v20191029
E3SM-1-1-ECA / r1i1p1f1 versions: v20191216, v20191218
EC-Earth3 / r1i1p1f1 versions: v20200312
EC-Earth3-LR / r1i1p1f1 versions: v20200409
EC-Earth3-Veg / r1i1p1f1 versions: v20200226
EC-Earth3-Veg-LR / r1i1p1f1 versions: v20200213
GISS-E2-1-G / r1i1p1f1 versions: v20180824
GISS-E2-1-G-CC / r1i1p1f1 versions: v20190815
GISS-E2-1-H / r1i1p1f1 versions: v20190410
GISS-E2-2-G / r1i1p1f1 versions: v20191120
HadGEM3-GC31-LL / r1i1p1f1 versions: v20190628
HadGEM3-GC31-MM / r1i1p1f1 versions: v20190920
INM-CM4-8 / r1i1p1f1 versions: v20190605
INM-CM5-0 / r1i1p1f1 versions: v20190619
IPSL-CM6A-LR / r1i1p1f1 versions: v20200326
MCM-UA-1-0 / r1i1p1f1 versions: v20190731, v20191017
MIROC6 / r1i1p1f1 versions: v20181212, v20190311
MPI-ESM-1-2-HAM / r1i1p1f1 versions: v20190627
MPI-ESM1-2-HR / r1i1p1f1 versions: v20190710
MPI-ESM1-2-LR / r1i1p1f1 versions: v20190710
MRI-ESM2-0 / r1i1p1f1 versions: v20190603, v20190222
NorCPM1 / r1i1p1f1 versions: v20190914
NorESM2-LM / r1i1p1f1 versions: v20200217, v20190920
NorESM2-MM / r1i1p1f1 versions: v20191108
SAM0-UNICON / r1i1p1f1 versions: v20190910
TaiESM1 / r1i1p1f1 versions: v20200302, v20200211

The results are displayed as a list of the model-members pairs followed by all the available versions.
Using the option --csv will produce a csv where, as we saw above, the paths and other attributes values will be listed too.
Similarly using the option --stats will summarise the results.

The same will work for --remote and/or cmip5. For the moment --and works only for --local or --remote queries and not for the standard query.
NB the attributes names can be different for different datasets, in this case variable for CMIP5 and variable_id for CMIP6.

!clef --remote cmip5 -v tasmin -v tasmax -e rcp26 -t day --and variable

CNRM-CM5 / r1i1p1 versions: v20121001
CSIRO-Mk3.6.0 / r10i1p1 versions: v20110518
CSIRO-Mk3.6.0 / r1i1p1 versions: v20120323
CSIRO-Mk3.6.0 / r2i1p1 versions: v20110518
CSIRO-Mk3.6.0 / r3i1p1 versions: v20110518
CSIRO-Mk3.6.0 / r4i1p1 versions: v20110518
CSIRO-Mk3.6.0 / r5i1p1 versions: v20110518
CSIRO-Mk3.6.0 / r6i1p1 versions: v20110518
CSIRO-Mk3.6.0 / r7i1p1 versions: v20110518
CSIRO-Mk3.6.0 / r8i1p1 versions: v20110518
CSIRO-Mk3.6.0 / r9i1p1 versions: v20110518
CanESM2 / r1i1p1 versions: v20130331
CanESM2 / r2i1p1 versions: v20130331
CanESM2 / r3i1p1 versions: v20130331
CanESM2 / r4i1p1 versions: v20130331, v20120410
CanESM2 / r5i1p1 versions: v20130331, v20120410
FGOALS-g2 / r1i1p1 versions: v20161204
GFDL-CM3 / r1i1p1 versions: v20110601
GFDL-ESM2G / r1i1p1 versions: v20130130
GFDL-ESM2M / r1i1p1 versions: v20110601
HadGEM2-AO / r1i1p1 versions: v20130815
HadGEM2-ES / r1i1p1 versions: v20120405, v20111206
HadGEM2-ES / r2i1p1 versions: v20111205
HadGEM2-ES / r3i1p1 versions: v20111208
HadGEM2-ES / r4i1p1 versions: v20111208
IPSL-CM5A-LR / r1i1p1 versions: v20120114
IPSL-CM5A-LR / r2i1p1 versions: v20130506
IPSL-CM5A-LR / r3i1p1 versions: v20111119
IPSL-CM5A-LR / r4i1p1 versions: v20120804
IPSL-CM5A-MR / r1i1p1 versions: v20111119
MIROC-ESM / r1i1p1 versions: v20120710
MIROC-ESM-CHEM / r1i1p1 versions: v20120710
MIROC5 / r1i1p1 versions: v20161012
MIROC5 / r2i1p1 versions: v20120710
MIROC5 / r3i1p1 versions: v20120710
MIROC5 / r4i1p1 versions: v20131009
MIROC5 / r5i1p1 versions: v20131009
MPI-ESM-LR / r1i1p1 versions: v20111014
MPI-ESM-LR / r2i1p1 versions: v20111014
MPI-ESM-LR / r3i1p1 versions: v20111014
MPI-ESM-MR / r1i1p1 versions: v20120503
MRI-CGCM3 / r1i1p1 versions: v20120701
NorESM1-M / r1i1p1 versions: v20110901

AND filter on more than one attribute#

We can passed the –and flag more than once if we want to apply the operator to more than one attribute.
Let’s add another experiment as a constraint to the previous CMIP6 query and see what happens.

!clef --local cmip6 -v pr -v mrso -e piControl -e ssp585 -mi r1i1p1f1 --frequency mon --and variable_id --and experiment_id

ACCESS-CM2 / r1i1p1f1 versions: v20191108, v20191112
ACCESS-ESM1-5 / r1i1p1f1 versions: v20191214, v20191115
BCC-CSM2-MR / r1i1p1f1 versions: v20181012, v20181016, v20190308, v20190314
CESM2-WACCM / r1i1p1f1 versions: v20190320, v20200702
CIESM / r1i1p1f1 versions: v20200605, v20200417
CMCC-CM2-SR5 / r1i1p1f1 versions: v20200616, v20200622
CanESM5 / r1i1p1f1 versions: v20190429
EC-Earth3 / r1i1p1f1 versions: v20200310, v20200312
EC-Earth3-Veg / r1i1p1f1 versions: v20200226, v20200225
INM-CM4-8 / r1i1p1f1 versions: v20190605, v20190603
INM-CM5-0 / r1i1p1f1 versions: v20190619, v20190724
IPSL-CM6A-LR / r1i1p1f1 versions: v20190903, v20200326
MIROC6 / r1i1p1f1 versions: v20190627, v20190311, v20181212
MPI-ESM1-2-HR / r1i1p1f1 versions: v20190710
MPI-ESM1-2-LR / r1i1p1f1 versions: v20190710
MRI-ESM2-0 / r1i1p1f1 versions: v20190603, v20191108, v20190222
NorESM2-LM / r1i1p1f1 versions: v20190920, v20200217, v20191108
NorESM2-MM / r1i1p1f1 versions: v20191108
TaiESM1 / r1i1p1f1 versions: v20200211, v20200901, v20200302

The query returned less results because, although we added an experiment, we are requiring that both variables and both experiments are available for a model/member pair to be selected.
By default model/member pair (or model/ensemble pairs for CMIP5) are used to identify a simulation. As we will see in the next and last blog when the same function is imported in a python code you can define the simulation using different attributes. The CORDEX dataset has more default attributes to define a simulation and they are different depending if you are doing a local or remote query. For local queries:
cordex_domain, driving_model_id, model_id and driving_model_ensemble_member
And for remote queries:
domain, driving_model, rcm_name and ensemble
These differences reflect the fact that the local query uses the file attributes, while the remote query uses the ESGF facets. Unfortunately different terminology have been used.

More tips on queries#

About experiment_family#

Experiment_family is a facet present only for CMIP5 and CORDEX. It allows you to select all the experiments following in the same category. The correspondent in CMIP6 is activity. However, not all experiments belong to a family and searching for both experiment and experiment_family at the same time can give unexpected results.
Let’s look at an example, if I want to get all the rcps experiments and historical I might be tempted to pass them as constraints in the same query:

!clef cmip5 -m CMCC-CM -e historical --experiment_family RCP -t Omon -v tos -en r1i1p1

ERROR: No matches found on ESGF, check at https://esgf.nci.org.au/search/esgf-nci?query=&type=File&distrib=True&replica=False&latest=True&project=CMIP5&ensemble=r1i1p1&experiment=historical&model=CMCC-CM&cmor_table=Omon&variable=tos&experiment_family=RCP

The ESGF query uses an AND operator for all the constraints we pass. We couldn’t find any matches because both the experiment and experiment_family constraints have to be satisfied. Historical is not part of the RCP family and so no results are returned. Similarly if we pass rcp45 as experiment as well as the family RCP we will only get the rcp45 results.

!clef cmip5 -m CMCC-CM -e rcp45 --experiment_family RCP -t Omon -v tos -en r1i1p1

/g/data/al33/replicas/CMIP5/combined/CMCC/CMCC-CM/rcp45/mon/ocean/Omon/r1i1p1/v20120518/tos/
/g/data/al33/replicas/CMIP5/combined/CMCC/CMCC-CM/rcp45/mon/ocean/Omon/r1i1p1/v20170725/tos/

Everything available on ESGF is also available locally

Finally, it is possible to use --experiment_family also in the local search:

!clef --local cmip5 -m CMCC-CM --experiment_family RCP -t Omon -v tos -en r1i1p1

/g/data/al33/replicas/CMIP5/combined/CMCC/CMCC-CM/rcp45/mon/ocean/Omon/r1i1p1/v20120518/tos
/g/data/al33/replicas/CMIP5/combined/CMCC/CMCC-CM/rcp45/mon/ocean/Omon/r1i1p1/v20170725/tos
/g/data/al33/replicas/CMIP5/combined/CMCC/CMCC-CM/rcp85/mon/ocean/Omon/r1i1p1/v20120528/tos
/g/data/al33/replicas/CMIP5/combined/CMCC/CMCC-CM/rcp85/mon/ocean/Omon/r1i1p1/v20170725/tos

Citation list option#

CleF has functions that retrieve errata associated to a file and the documents available in the ESDOC system.
Most of these functionalities are currently working only when you use clef interactively. However, since version 1.2 we added a flag to retrieve citations also using the command line.
The --cite option added to the command line will create a file containing the citations of all the datasets returned by the query. It retrieves the citation information from the DKRZ WDCC server (https://cera-www.dkrz.de/WDCC). This provides citation information only for CMIP6, so this flag is only available with the cmip6 sub-command.
As for the other flags illustrated above, this option works when you select either --local or --remote queries.

!clef --remote cmip6 -v clt  -e historical -t day  -mi r1i1p1f1 --cite

CMIP6.CMIP.AS-RCEC.TaiESM1.historical.r1i1p1f1.day.clt.gn.v20200626
CMIP6.CMIP.AWI.AWI-ESM-1-1-LR.historical.r1i1p1f1.day.clt.gn.v20200212
CMIP6.CMIP.BCC.BCC-CSM2-MR.historical.r1i1p1f1.day.clt.gn.v20181127
CMIP6.CMIP.BCC.BCC-ESM1.historical.r1i1p1f1.day.clt.gn.v20181220
CMIP6.CMIP.CAS.FGOALS-f3-L.historical.r1i1p1f1.day.clt.gr.v20200605
CMIP6.CMIP.CAS.FGOALS-g3.historical.r1i1p1f1.day.clt.gn.v20190826
CMIP6.CMIP.CCCR-IITM.IITM-ESM.historical.r1i1p1f1.day.clt.gn.v20191226
CMIP6.CMIP.CCCma.CanESM5.historical.r1i1p1f1.day.clt.gn.v20190429
CMIP6.CMIP.CMCC.CMCC-CM2-HR4.historical.r1i1p1f1.day.clt.gn.v20200904
CMIP6.CMIP.CMCC.CMCC-CM2-SR5.historical.r1i1p1f1.day.clt.gn.v20200616
CMIP6.CMIP.CSIRO-ARCCSS.ACCESS-CM2.historical.r1i1p1f1.day.clt.gn.v20191108
CMIP6.CMIP.CSIRO.ACCESS-ESM1-5.historical.r1i1p1f1.day.clt.gn.v20191115
CMIP6.CMIP.EC-Earth-Consortium.EC-Earth3-Veg-LR.historical.r1i1p1f1.day.clt.gr.v20200217
CMIP6.CMIP.EC-Earth-Consortium.EC-Earth3-Veg.historical.r1i1p1f1.day.clt.gr.v20200225
CMIP6.CMIP.EC-Earth-Consortium.EC-Earth3.historical.r1i1p1f1.day.clt.gr.v20200310
CMIP6.CMIP.HAMMOZ-Consortium.MPI-ESM-1-2-HAM.historical.r1i1p1f1.day.clt.gn.v20190627
CMIP6.CMIP.INM.INM-CM4-8.historical.r1i1p1f1.day.clt.gr1.v20190530
CMIP6.CMIP.INM.INM-CM5-0.historical.r1i1p1f1.day.clt.gr1.v20190610
CMIP6.CMIP.IPSL.IPSL-CM6A-LR.historical.r1i1p1f1.day.clt.gr.v20180803
CMIP6.CMIP.KIOST.KIOST-ESM.historical.r1i1p1f1.day.clt.gr1.v20191202
CMIP6.CMIP.MIROC.MIROC6.historical.r1i1p1f1.day.clt.gn.v20191016
CMIP6.CMIP.MPI-M.MPI-ESM1-2-HR.historical.r1i1p1f1.day.clt.gn.v20190710
CMIP6.CMIP.MPI-M.MPI-ESM1-2-LR.historical.r1i1p1f1.day.clt.gn.v20190710
CMIP6.CMIP.MRI.MRI-ESM2-0.historical.r1i1p1f1.day.clt.gn.v20190603
CMIP6.CMIP.NCAR.CESM2-FV2.historical.r1i1p1f1.day.clt.gn.v20191120
CMIP6.CMIP.NCAR.CESM2-WACCM-FV2.historical.r1i1p1f1.day.clt.gn.v20191120
CMIP6.CMIP.NCAR.CESM2-WACCM.historical.r1i1p1f1.day.clt.gn.v20190227
CMIP6.CMIP.NCAR.CESM2.historical.r1i1p1f1.day.clt.gn.v20190308
CMIP6.CMIP.NCC.NorESM2-LM.historical.r1i1p1f1.day.clt.gn.v20190815
CMIP6.CMIP.NCC.NorESM2-MM.historical.r1i1p1f1.day.clt.gn.v20191108
CMIP6.CMIP.NIMS-KMA.KACE-1-0-G.historical.r1i1p1f1.day.clt.gr.v20190911
CMIP6.CMIP.NOAA-GFDL.GFDL-CM4.historical.r1i1p1f1.day.clt.gr1.v20180701
CMIP6.CMIP.NOAA-GFDL.GFDL-CM4.historical.r1i1p1f1.day.clt.gr2.v20180701
CMIP6.CMIP.NUIST.NESM3.historical.r1i1p1f1.day.clt.gn.v20190812
CMIP6.CMIP.SNU.SAM0-UNICON.historical.r1i1p1f1.day.clt.gn.v20190323

The citations are listed in a cmip_citations.txt file in the current directory.

! head -n 4 cmip_citations.txt 

Lee, Wei-Liang; Liang, Hsin-Chien (2020). AS-RCEC TaiESM1.0 model output prepared for CMIP6 CMIP historical. Version v20200626. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.9755
Danek, Christopher; Shi, Xiaoxu; Stepanek, Christian; Yang, Hu; Barbi, Dirk; Hegewald, Jan; Lohmann, Gerrit (2020). AWI AWI-ESM1.1LR model output prepared for CMIP6 CMIP historical. Version v20200212. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.9328
Wu, Tongwen; Chu, Min; Dong, Min; Fang, Yongjie; Jie, Weihua; Li, Jianglong; Li, Weiping; Liu, Qianxia; Shi, Xueli; Xin, Xiaoge; Yan, Jinghui; Zhang, Fang; Zhang, Jie; Zhang, Li; Zhang, Yanwu (2018). BCC BCC-CSM2MR model output prepared for CMIP6 CMIP historical. Version v20181127. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.2948
Zhang, Jie; Wu, Tongwen; Shi, Xueli; Zhang, Fang; Li, Jianglong; Chu, Min; Liu, Qianxia; Yan, Jinghui; Ma, Qiang; Wei, Min (2018). BCC BCC-ESM1 model output prepared for CMIP6 CMIP historical. Version v20181220. Earth System Grid Federation. https://doi.org/10.22033/ESGF/CMIP6.2949

Please note that some of the more recently published datasets might not have citations information available yet on the WDCC server. CleF retrieves the information from different fields and them put them together to form a correct citation in the required format. If some of these fields are not available the citation might be incomplete, so always check the entire file before using it.
See below for an example of this:

! clef --local cmip6 --model TaiESM1 --experiment ssp585 --variable pr --table Amon --cite
!head -n 4 cmip_citations.txt

/g/data/oi10/replicas/CMIP6/ScenarioMIP/AS-RCEC/TaiESM1/ssp585/r1i1p1f1/Amon/pr/gn/v20200901
 (2018). AS-RCEC TaiESM1.0 model output prepared for CMIP6 ScenarioMIP ssp585. Earth System Grid Federation. http://cera-www.dkrz.de/WDCC/meta/CMIP6/CMIP6.ScenarioMIP.AS-RCEC.TaiESM1.ssp585