TPC5 File Format

From Elsys Wiki - Help and Recources for TranAX and TraNET DAQ
Jump to: navigation, search


TranAX is saving trace date, spectral data and TabPages in the HDF5 file format. Depending on the data type the file extention is *.tpc5, *.tps5 or *.tpd.

The HDF5 file format was developed at the University of Illinois. The HDF5 specifications and the libraries source codes are open.

Specification


Matlab Data Import

More information about HDF5 support in Matlab can be found on the MathWorks Website.

Download the example below, execute the script startdemo.m. This will open an plot multiple traces from a tpc5 file. Please note this example will use a file, saved from a Scope measurement in TranAX. For Multiblock and ECR support, this example hast to be extended.

tpc5Trace.m

% Matlab Example, read data from tpc5 file
% Copyright: 2018 Elsys AG
% Author: 2011-01-18, R. Bertschi
% Rev. 2018-03-12, Thomas Berger
% Description:
% This example shows how to read one trace from a TPC5 file
% and the conversion to physical values

function [timeAxis, dataPhysical, traceInfo] = loadtpc5(filename, nr)

% clear data arrays
clear timeAxis;
clear dataPhysical;
clear traceInfo;

% strings to hd5 path, access data
StrTpc5Chn = strcat('/measurements/00000001/channels/', sprintf('%08d',nr));
StrTpc5Blk = strcat(StrTpc5Chn, '/blocks/00000001/');
tpc5ChnRaw = strcat(StrTpc5Chn, '/blocks/00000001/raw');

% Read out channel settings for the first channel
% Digital marker signals are a part of the binary measurement values.
% For separating digital from analog values the following two
% bit masks are needed
analogMask = uint16( h5readatt(filename,StrTpc5Chn, 'analogMask'));
markerMask = uint16( h5readatt(filename,StrTpc5Chn, 'markerMask'));


% Measurement values are stored as they are coming from the A/D converter.
% For getting voltage values or scaled physical values the corresponding
% conversion values must be read out.
% Conversion factor for scaling to voltage values
binToVoltFactor         = h5readatt(filename,StrTpc5Chn, 'binToVoltFactor');
binToVoltConstant       = h5readatt(filename,StrTpc5Chn, 'binToVoltConstant');
% Conversion factor for scaling to physical values
voltToPhysicalFactor    = h5readatt(filename,StrTpc5Chn, 'voltToPhysicalFactor');
voltToPhysicalConstant  = h5readatt(filename,StrTpc5Chn, 'voltToPhysicalConstant');

% Read out the data from the first channel, first block
data = h5read(filename, tpc5ChnRaw);
% Mask out the digital marker bits
analogData = bitand(data,analogMask);

% Scale data to voltage values
dbData = cast(analogData,'double');
dataScaled = cast((dbData * binToVoltFactor) + binToVoltConstant,'double');

% Scale data to physical values
dataPhysical = (dataScaled * voltToPhysicalFactor) + voltToPhysicalConstant;

% Recording parameters
sampleRate  = h5readatt(filename,StrTpc5Blk, 'sampleRateHertz');
triggerSample = h5readatt(filename,StrTpc5Blk, 'triggerSample');

% number of samples
nSamples = length(dataPhysical);
% time between each sample
tSample = double(1/sampleRate);

% Trigger position
tTrigger = double (triggerSample) * double(tSample);

% Recording length
tRecord = nSamples * tSample;
tBegin = -tTrigger;
tEnd = tRecord - tTrigger;

% X-axis (time)
timeAxis = linspace(tBegin, tEnd, nSamples)';

% create struct with trace information
tmp = h5readatt(filename,StrTpc5Chn, 'name');  % returns type 'cell'
traceInfo.name = tmp{1}; % get item as string
tmp = h5readatt(filename,StrTpc5Chn, 'physicalUnit');  % returns type 'cell'
traceInfo.physicalUnit = tmp{1}; % get item as string
traceInfo.sampleRateHertz  = h5readatt(filename,StrTpc5Blk, 'sampleRateHertz');
traceInfo.triggerSample = h5readatt(filename,StrTpc5Blk, 'triggerSample');
traceInfo.nSamples = nSamples;

return

tpc5Info.m

% Matlab Example, read number of traces from tpc5 file
% Copyright: 2018 Elsys AG
% Author: 2018-03-12, Thomas Berger
% Description:
% Return the number of traces in a tpc5 file


function [nrofTraces] = fileInfoTpc5(filename)
	nrofTrace=0;

	% Read the file structure, just for get an overview
	n = h5info(filename, '/measurements/00000001/channels/');
	[nrofTraces,tmp] = size( n.Groups);
return

tpc5Plot.m

% Matlab Example, plot multiple traces from tpc5 file
% Copyright: 2018 Elsys AG
% Author: 2018-03-12, Thomas Berger
% Description:
% This example shows how to plot traces

function  tpc5Plot(x, C,opt,n,fig,all)
% define plot area and clear
figure(fig);
clf;

if (all==true)
    % All traces into the same plot
    plot(x,C);
    labeltpcplot(opt,n,1,n);
else
    % Each traces into a single plot
    for z=1:n
        subplot(n,1,z);
        plot(x,C(z,:));
        labeltpcplot(opt,n,z,z);
    end
end
return

% Label both axes, create legend with traces
function labeltpcplot(opt,n,chnfrom,chnto)

% Y-Label definition
y='';
for z=chnfrom:chnto
    y =[y,'[',opt(z).physicalUnit,'] '];
end

% define legend
clear l;
for z=chnfrom:chnto
    tmp= [int2str(z),': "',opt(z).name,'" [',opt(z).physicalUnit,'] '];   
    if (z==chnfrom)
        l= tmp;
    else
        l=char(l,tmp);
    end
end

% Write title on top of the first plot
if (chnfrom==1)
    title([int2str(n),' Trace(s)']);
end;
xlabel('Seconds')
ylabel(y);
legend(l);

return

startdemo.m

% Matlab Example, read multiple traces from tpc5 file
% Copyright: 2018 Elsys AG
% Author: 2018-03-12, Thomas Berger
% Description:
% This example shows how to read and plot traces from a TPC5 file

% Filename and path to Elsys tpc5 file
filename = 'mytrace.tpc5';

% Read number of traces and channel information
nTraces= tpc5Info(filename);

% loop through each channel
for n = 1:nTraces
    % one single trace
    [x, y, traceInfo] = tpc5Trace(filename, n);
    
    % build an array with all Y values
    C(n,:)=y;
    opt(n,:)=traceInfo;
end

% plot all traces into one plot
tpc5Plot(x, C, opt, nTraces, 1, true);

% plot traces in areas
tpc5Plot(x, C, opt, nTraces, 2, false);


Python Example

TPC5.py

''' Get Raw Dataset Block'''
def getDataSetName(channel, block = 1):
    blockString   = "%08d" % block
    channelString = "%08d" % channel
    name = '/measurements/00000001/channels/' + channelString + '/blocks/' + blockString + '/raw'
    return name

def getChannelGroupName(channel):
    channelString = "%08d" % channel   
    return '/measurements/00000001/channels/' + channelString +'/'

def getBlockName(channel, block):
    blockString   = "%08d" % block
    channelString = "%08d" % channel
    name = '/measurements/00000001/channels/' + channelString + '/blocks/' + blockString + '/'
    return name

def getVoltageData(fileRef, channel, block = 1):
    channel_group           = fileRef[getChannelGroupName(channel)]
    dataset_name            = getDataSetName(channel,block)

    ''' Get Scaling Parameters '''
    binToVoltageFactor      = channel_group.attrs['binToVoltFactor']
    binToVoltageConstant    = channel_group.attrs['binToVoltConstant']

    ''' Get Analog and Digital Mask for Data separation '''
    analogMask              = channel_group.attrs['analogMask']
    markerMask              = channel_group.attrs['markerMask']

    analogData              = fileRef[dataset_name] & analogMask
    
    ''' Scale To voltage '''
    return analogData * binToVoltageFactor + binToVoltageConstant

def getPhysicalData(fileRef, channel, block = 1):
    channel_group           = fileRef[getChannelGroupName(channel)]
    dataset_name            = getDataSetName(channel,block)

    ''' Get Scaling Parameters '''
    binToVoltageFactor      = channel_group.attrs['binToVoltFactor']
    binToVoltageConstant    = channel_group.attrs['binToVoltConstant']
    VoltToPhysicalFactor    = channel_group.attrs['voltToPhysicalFactor']
    VoltToPhysicalConstant  = channel_group.attrs['voltToPhysicalConstant']

    ''' Get Analog and Digital Mask for Data separation '''
    analogMask              = channel_group.attrs['analogMask']
    markerMask              = channel_group.attrs['markerMask']
    
    analogData              = fileRef[dataset_name] & analogMask
    
    ''' Scale To voltage '''
    voltageData = analogData * binToVoltageFactor + binToVoltageConstant
    return voltageData * VoltToPhysicalFactor + VoltToPhysicalConstant

def getChannelName(fileRef, channel):
    channel_group           = fileRef[getChannelGroupName(channel)]
    return channel_group.attrs['name']

def getPhysicalUnit(fileRef, channel):
    channel_group           = fileRef[getChannelGroupName(channel)]
    return  channel_group.attrs['physicalUnit']

def getSampleRate(fileRef, channel, block = 1):
    block_group             = fileRef[getBlockName(channel,block)]
    return block_group.attrs['sampleRateHertz']

def getTriggerSample(fileRef, channel, block = 1):
    block_group             = fileRef[getBlockName(channel,block)]
    return block_group.attrs['triggerSample']

def getTriggerTime(fileRef, channel, block = 1):
    block_group             = fileRef[getBlockName(channel,block)]
    return block_group.attrs['triggerTimeSeconds']

def getStartTime(fileRef, channel, block = 1):
    block_group             = fileRef[getBlockName(channel,block)]
    return block_group.attrs['startTime']
import h5py
import numpy as np
from pylab import *

''' Import Helper Function for reading TPC5 Files '''
import tpc5


f = h5py.File("TestData.tpc5", "r")

''' Get Data scaled int voltage from channel 1 '''
dataset1 = tpc5.getVoltageData(f,1)
''' Get Data scaled in physical unit from channel 2'''
dataset2 = tpc5.getPhysicalData(f,2)


ch1 = tpc5.getChannelName(f, 1)
ch2 = tpc5.getChannelName(f, 2)

''' Build Unit String '''
unit = '[ ' + tpc5.getPhysicalUnit(f,1) + ' ]'

''' Get Time Meta Data TriggerSample number and Sampling Rate '''
TriggerSample = tpc5.getTriggerSample(f,1,1)
SamplingRate  = tpc5.getSampleRate(f,1,1)

''' Get Absolute recording time and split it up '''
RecTimeString = tpc5.getStartTime(f,1,1)
RecTimeList   = RecTimeString.split('T',1)
RecDate       = RecTimeList[0]
TimeListe     = RecTimeList[1].split('.',1)
RecTime       = TimeListe[0] 

f.close

''' Make Plot '''
fig = plt.figure()
fig.suptitle('Imported Tpc5 File', fontsize=14, fontweight='bold')

''' Scale x Axis to ms '''
TimeScale = 1000

''' Build up X-Axis Array '''
startTime = -TriggerSample /SamplingRate * TimeScale
endTime   = (len(dataset1)-TriggerSample)/SamplingRate * TimeScale
t         = arange(startTime, endTime, 1/SamplingRate * TimeScale)


plot(t, dataset1, label=ch1)
plot(t, dataset2, label=ch2)

legend(framealpha=0.5)

if TimeScale == 1: 
    xlabel('time (s)')
elif TimeScale == 1000:
    xlabel('time (ms)')
elif TimeScale == 1000000:
    xlabel('time (us)')

ylabel(unit)

title('Recording Time: ' + RecDate + ' ' + RecTime)

grid(True)
savefig("TestDataPlot.png")
show()

C++ Reading Example

#include "stdafx.h"
#include <iostream>
#include <fstream>

using std::cout;
using std::endl;
using std::ofstream;

#include <string>
#include "H5Cpp.h"		// Using the C++ High Level API
using namespace H5;


int _tmain(int argc, _TCHAR* argv[])
{

	unsigned int const BLOCK_SIZE = 256 * 1024;

	try {
		H5File file("Example.tpc5", H5F_ACC_RDONLY);

		// Read out channel settings for the first channel

		// Use HDFView for further analysis of the file structure of your file

		// Digital marker signals are a part of the binaray measurement values.
		// For separating digital from analog values the following two
		// bit maskes are needet (is the same for all channel of one board)
		Group g1 = file.openGroup("/measurements/00000001/channels/00000001");
		Attribute ch1_analogmask = g1.openAttribute("analogMask");

		unsigned int analogmask_ch1;
		ch1_analogmask.read(PredType::NATIVE_INT, &analogmask_ch1);
		ch1_analogmask.close();

		Attribute ch1_markermask = g1.openAttribute("markerMask");
		unsigned int markermask_ch1;
		ch1_markermask.read(PredType::NATIVE_INT, &markermask_ch1);
		ch1_markermask.close();

		// Measurement values are stored as they are comming from the A / D converter.
		// For getting voltage values or scaled physical values the corresponding
		// convertion values must be read out. (could be different for each channel)
		Attribute ch1_binToVoltFactor = g1.openAttribute("binToVoltFactor");
		Attribute ch1_binToVoltConstant = g1.openAttribute("binToVoltConstant");
		float binToVoltFactor_ch1, binToVoltConstant_ch1;
		ch1_binToVoltFactor.read(PredType::NATIVE_FLOAT, &binToVoltFactor_ch1);
		ch1_binToVoltConstant.read(PredType::NATIVE_FLOAT, &binToVoltConstant_ch1);
		ch1_binToVoltFactor.close();
		ch1_binToVoltConstant.close();

		// Conversion factor for scaling to physical values
		Attribute ch1_voltToPhysicalFactor = g1.openAttribute("voltToPhysicalFactor");
		Attribute ch1_voltToPhysicalConstant = g1.openAttribute("voltToPhysicalConstant");
		float voltToPhysicalFactor_ch1, voltToPhysicalConstant_ch1;
		ch1_voltToPhysicalFactor.read(PredType::NATIVE_FLOAT, &voltToPhysicalFactor_ch1);
		ch1_voltToPhysicalConstant.read(PredType::NATIVE_FLOAT, &voltToPhysicalConstant_ch1);
		ch1_voltToPhysicalFactor.close();
		ch1_voltToPhysicalConstant.close();

		// Read out the data from the first channel, first block
		DataSet dataset_ch1_block1 = file.openDataSet("/measurements/00000001/channels/00000001/blocks/00000001/raw");

		int* rawData = new int[BLOCK_SIZE];
		dataset_ch1_block1.read(rawData, PredType::NATIVE_INT);
		dataset_ch1_block1.close();

		// Mask out the digital marker bits
		int* analogData = new int[BLOCK_SIZE];
		int* markerData = new int[BLOCK_SIZE];
		float* voltageData = new float[BLOCK_SIZE];
		float* physicalData = new float[BLOCK_SIZE];

		// Output Text file
		ofstream myfile;
		myfile.open("example.txt");

		for (int i = 0; i < BLOCK_SIZE; i++){
			analogData[i] = rawData[i] & analogmask_ch1;
			markerData[i] = rawData[i] & markermask_ch1;

			// Scale Data to voltage and/or physical data
			voltageData[i] = ((float)analogData[i] * binToVoltFactor_ch1) + binToVoltConstant_ch1;
			physicalData[i] = (voltageData[i] * voltToPhysicalFactor_ch1) + voltToPhysicalConstant_ch1;

			// Log Data
			myfile << physicalData[i] << endl;
		}

		myfile.close();
		g1.close();
		file.close();

		delete rawData;
		delete analogData;
		delete markerData;
		delete voltageData;
		delete physicalData;
	}
	catch (AttributeIException error)
	{
		error.printError();
		return -1;
	}
	catch (...){}
	return 0;
}