Search

CDEF Data Format

You are here:

Date: 24 Jan 2005

CARBOEUROPE-IP

TASK 1.2.1, 1.2.2
Matthias Mauder, Thomas Foken
University of Bayreuth

Exchange format for high frequency raw data

Input format: ASCII
Columns are separated by commas.
For the number format decimal places are separated by a dot (1.23456)
Missing values are indicated by [cci_text]-9999.9[/cci_text]

High frequency raw data

Coordinate system

Wind components u, v, w are defined in a right hand coordinate system.

File header information like this example:

[cc_text]

CARBOEUROPE high frequency data exchange format
site: DE-Wei
time used: UTC
Name of responsible person: Matthias Mauder
Sonic type: CSAT3 **)
Analyser type: LI-7500 ***)
measuring height above ground (m): 2.70
canopy height (m): 0.50
orientation of the u-component (0-360): 220
Height above sea level (m): 72
Latitude (deg,min,sec): 52,13,55
Year of measurement: 2003
sampling frequency (Hz): 20
orientation of analyser against sonic (0-360): 220
sensor separation sonic – analyser (m): 0.30
sensor separation add. fast temperature sensor (m)*): 0.05
time constant of add. fast temperature sensor (s)*): 0.02
DOY,HHMM,SEC,u(m/s),v(m/s),w(m/s),Ts(C),Tp(C),a(g/m3),CO2(mmol/m3)
121,1200,00.10, 3.455,-0.123, 0.045, 18.18, 17.72, 8.423, 15.371
*) sensor type not available at all sites, line is optional
**) please use only: CSAT3, USA-1, Solent-HS, Solent-R2, Solent-R3, ATI-K,
Young
***) please use only LI-7500, LI-6262, LI-7000, KH20

[/cc_text]

The data producer can decide about the decimal places used in the data file. The length of the data files is also free. 30 minutes, 60 minutes or 24 hours, everything is possible. The header line directly above the first data line has to be like this. It is meant to give information to the user of the data to recognize the structure of the data. But this line is ignored by the program. The columns have to be exactly in the given order. No column can be skipped. 10 columns with the parameters in the given order are required. If the measurements of one column eventually are not available, fill it with -9999.9 .

All lines of the the header are necessary apart from the last two lines indicated with *) in the given order. The header parameters (site, time etc.) are read by the program automatically. Therefore, it is important that every single letter is the same like in the example given above.

If the configuration parameters don’t change in the processing period, the header can be omitted. Then the required parameters have to be given separately from the data file. This should be an exception.

Convention about the file names:

[cci_text]SSSSSS_H####.dat[/cci_text]

[cci_text]SSSSSS[/cci_text]: abbreviation code of the station, e.g. DE-Wei for Waldstein/Weidenbrunnen
[cci_text]####[/cci_text]: running number of the files, please number the data files continuously (4 digits). When the current file is read the program will always look for the file with the
following number.

Low frequency reference data

Accurate reference data for temperature, humidity and pressure are necessary for meteorological calculations during the eddy covariance data post processing. These
are usually averaged over time periods like 5 or 10 minutes.

File header information like this example:

[cc_text]

CARBOEUROPE low frequency data exchange format
site: DE-Wei
time used: UTC
Name of responsible person: Matthias Mauder
measuring height above ground (m): 2.70
canopy height (m): 0.50
DOY,HHMM,SEC(begin),DOY,HHMM,SEC(end),T_ref(C),a_ref(g/m3),p_ref(hPa)
121,1200,00,121,1210,00,19.532,10.7992,1013.0

[/cc_text]

Convention about the file names:

[cci_text]SSSSSS_L####.dat[/cci_text]

[cci_text]SSSSSS[/cci_text]: abbreviation code of the station, e.g. DE-Wei for Waldstein/Weidenbrunnen
[cci_text]####[/cci_text]: running number of the files, please number the data files continuously.

5 min Averaged Data (Task 1.2.1 only)

Raw data files are often useful to find reason for eventual errors. But if you consider it to cause too much effort to create the raw data files, we offer the alternative to send averaged data for the footprint and quality assessment of main flux sites (Task 1.2.1). These files contain averages, variances and covariances for 5 minute intervals. It is necessary to have averages not longer than 5 minutes to make it possible to perform the quality test. 30 minutes would be definitely too long.

Coordinate system

Wind components u, v, w are defined in a right hand coordinate system.

File header information like this example:

[cc_text]

CARBOEUROPE averaged data exchange format
site: DE-Wei
time used: UTC
Name of responsible person: Matthias Mauder
Sonic type: CSAT3 **)
Analyser type: LI-7500 ***)
measuring height above ground (m): 2.70
canopy height (m): 0.50
orientation of the u-component (0-360): 220
Height above sea level (m): 72
Latitude (deg,min,sec): 52,13,55
Year of measurement: 2003
sampling frequency (Hz): 20
orientation of analyser against sonic (0-360): 220
sensor separation sonic – analyser (m): 0.30
sensor separation add. fast temperature sensor (m)*): 0.05
time constant of add. fast temperature sensor (s)*): 0.02

[/cc_text]

see also EXAMPLE_5M001.CSV

Convention about the file names:

[cci_text]SSSSSS_5M###.CSV[/cci_text]

[cci_text]SSSSSS[/cci_text]: abbreviation code of the station, e.g. DE-Wei for Waldstein/Weidenbrunnen
[cci_text]5M[/cci_text]: for 5 minute averaging interval
[cci_text]###[/cci_text]: running number of the files, please number continuously
[cci_text]CSV[/cci_text]: file suffix

Format:

Header line with identifier for the quantity and its unit.

Dot as decimal separator, columns comma separated, [cci_text]-9999.9[/cci_text] as identifier for wrong or missing values.

The order of the columns must be in the same way like in the example file EXAMPLE_5M001.CSV. If the quantities, which are listed in a column, are not measured, please fill up these columns with -9999.9. For example the columns for low frequency reference measurements can be filled up with -9999.9, if these measurements are not available. These reference data attribute to the confidence of the analysis. But in case they are not available the analysis can be performed without them. In order to be able to calculate averages, variances and covariances, it is necessary to have information about the specific number of values.

DOY,HHMM,SEC(begin): Beginning of the averaging interval in UTC
DOY,HHMM,SEC(end): Ending of the averaging interval in UTC
u(m/s): horizontal wind component for the direction in which the sonic is oriented
v(m/s): horizontal wind component for the direction rectangular to the orientation of
the sonic
w(m/s): vertical wind component
Ts(C): sonic temperature
Tp(C): temperature from an additional fast response sensor, PT150 or Fine Wire
Thermocouple
a(g/m3): absolute humidity averaged from turbulence measurement
CO2(mmol/m3): CO2 concentration averaged from turbulence measurement
T_ref(C): temperature measurement from a slow response reference sensor
a_ref(g/m3): absolute humidity from a slow response reference sensor
p_ref(hPa): air pressure
Var(u), Var(v), Var(w): Variance of the wind components
Var(Ts): Variance of sonic temperature
Var(Tp): Variance of Temperature from a additional fast response sensor
Var(a): Variance of absolute humidity
Var(CO2): Variance of CO2 concentration
Cov(u’v’): Covariance between the wind components u and v
Cov(v’w’): Covariance between the wind components v and w
Cov(u’w’): Covariance between the wind components u and w
Cov(u’Ts’), Cov(v’Ts’), Cov(w’Ts’): Covariance between the three wind components
and the sonic temperature
Cov(u’Tp’), Cov(v’Tp’), Cov(w’Tp’): Covariance between the three wind components
and the temperature from an additional fast response sensor
Cov(u’a’), Cov(v’a’), Cov(w’a’): Covariance between the three wind components and
absolute humidity
Cov(u’CO2′), Cov(v’CO2′), Cov(w’CO2′): Covariance between the three wind
components and the CO2 concentration
N(u), N(v), N(w), N(Ts), N(Tp), N(a), N(CO2): Number of values for the respective
quantity
N(u’v’), N(v’w’), N(u’w’), N(u’Ts’), N(v’Ts’), N(w’Ts’), N(u’Tp’), N(v’Tp’), N(w’Tp’),
N(u’a’), N(v’a’), N(w’a’), N(u’CO2′), N(v’CO2′), N(w’CO2′): Number of values going into
the calculation of the respective covariance

Download

CDEF_Data_Format.pdf