5. Plug-ins

5.1. Introduction.

Building applications mostly involves putting together parts that are commonly used and a specific part responsible for the 'new' approach being investigated. To simplify reuse of parts that previously have been developed, BrainStream? supports plug-ins. The common parts then can be added to the new application as plug-ins and do not need to be redefined. It saves the researcher a lot of development time and encourages sharing parts of experiments between researchers within or among institutes. Although a plug-in can also be a function inserted in the function pipe-line, here it is focused on plug-in tables. Compared to function-only plug-ins, table plug-ins define a lot more, i.e. the specification of actions for multiple markers at different timepoints.

Plug-in tables can be used in two different ways:
1) independent: the plug-in table entirely defines a complete implementation of some useful feature and has all required information defined.
2) dependent: the plug-in table actions must be merged with actions of (an)other marker(s), therefore, always a substitute needs to be defined. Information from the other merged marker(s) is required, they become ONE event and concequently information can be shared via the event structure.

5.2. Independent plug-ins.

These plug-ins define all that is necessary to execute. Just add the reference to this plug-in in your experiment definition table and all actions are available. It is useful since this way a collection of markers' actions that are often used can be added to your experiment at once. Another way of using these plug-ins is to define a lot of same/default actions for a list (large number of) of markers.

5.3. Dependent plug-in tables.

Dependent plug-ins are different in the way they are incorporated in the experiment definition. Since they rely on information from other markers it is required to specify which. This can be done by definition of a substitute for the dependent marker in this plug-in, that is the one that depends on information from other marker(s). The substitute tells BrainStream? to replace the incoming marker with the markers specified in this substitute. For instance if the substitute defines: m1=m0,m1,m2, then for every incoming marker m1, BrainStream? actually inserts the markers m0, m1, and m2. If m1 defines a DATA timepoint, it assumes one of the substitutes (m0 or m2) defines what data segment is required and thus m1 'inherits' their dataselection information. In general, also other information should be supplied by one of the substituted markers. The plug-in table should tell what type of information should be delivered, i.e., it defines its dependencies. For example, if a plug-ins purpose is to plot a signal, it would need the data to be plotted. The plug-in table should mention something like: 'this plug-in depends on event.plot.data (channels x data)', it should also define possible parameters that can be used to change output behaviour.

Dependent plug-in tables serve to add partial functionality to another plug-in table (or main table), for instance plotting a signal is very often used to visualize relevant outcome of processed data. Regardless of what content is being plotted, the steps required to get it visualized are very similar in every case. Replacing this last part by a plug-in saves a lot of time during setting up the experiment definition tables. Adding such plug-in now only requires a reference in the main table, a definition of the substitute and in the code a line that defines the data to be plotted. Also, if non-default parameters should apply to the plug-in, set keys for corresponding plug-ins topic in the block settings file accordingly.
BrainStream will examine the substituted markers for efficiency reasons and adapt actions accordingly. For instance, if two successive markers in the substitute define a 'get' action on the same variable (equal timepoint!) the latter one will automatically be removed by BrainStream? since it becomes meaningless in this particular substitute. The substitute might also reduce the amount of data being transported internally in BrainStream? since variables content remain available among the substituted markers and don't need to be copied around from and to the user state (user workspace), which would be the case if non-substituted markers exchange user workspace information.

If substituted markers are executed parallel-wise, 'lock' information is also updated accordingly since the substitute influences decisions on what can execute in parallel.

Below, an example of a simple application using the topoplot plug-in.

Topoplot example (in file: mybci.xls, sheet Actions)
markertimefunctionclienthdr
flashDATApreProc, getBandPower get
@topoplot.xls Topoplot

*Topoplot example* (in file: mybci.xls, sheet Dictionary)
markertypevaluesubstitute
flashstimulus21flash, update_topoplot
update_topoplotstimulus22

Topoplot example (sheet DataSelection)
marker begintime endtime
flash 0 1

*Topoplot plug-in* (in file: topoplot.xls, sheet Topoplot)
markertimefunctionclienthdrtopoplot
BS_INITEVENTbs_insert_marker('init_topoplot',0)
init_topoplotEVENTinit_topoplottopoplotget[],get,put
update_topoplotDATAupdate_topoplottopoplotgetget, put

% make sure that your pre-executed user functions define content for event.topoplot.data (which will be plotted)
% make sure you define a substitute in the dictionary for the marker to which update_topoplot should be merged!
% make sure you define topoplot client in your block settings (set to 'local' if you are not running in parallel mode)
% make sure the lib folder is included in the Matlab path!
% you do not need to add gets or puts for the topoplot variable, this is handled by the topoplot plug-in
 

The first 3 tables demonstrate what is needed to add the plug-in to your application. In the main Actions sheet add the reference: @topoplot.xls Topoplot to the plug-in table in order to add all corresponding actions to your application. Define the substitute by first adding a third column substitute in the Dictionary sheet and then give a comma seperated list of markers that need to be substituted. In this case, at every occurence of the flash marker, BrainStream? will insert the flash AND subsequently the update_topoplot marker. Using the same marker in the substitute is not obligate, you could also use all different markers in the substitute list. Be careful doing this, since it will confuse manual reading of the tables. The definition of the requested data is required anyway, this information is inherited by the plug-in. Last step is to make sure the topoplot plug-in is provided with correct information, in the getBandPower function a statement is included: event.topoplot.data = rand(nchans,1); In this example only filled with random numbers. This information will then be used by the topoplot plug-in to plot the data. For this plug-in, it is also possible to change some parameters by defining topic and key-values in the block settings, see topoplot plug-in.

The last table shows how the topoplot plug-in is defined. Initialization of the plug-in is achieved by inserting a marker init_topoplot as soon as the BS_INIT marker is processed. This way it is guaranteed that the figure object is instantiated at the right place, i.e., at the client actual executing the functions. Although this plug-in defines a DATA timepoint for the update_topoplot marker, it doesn't define any data selection; this information will be inherited from the other substitute markers. The hdr variable will always be initialized by BrainStream. Set Clients.topoplot to 'local' in order to execute functions serial-wise, or set it to a client if it should run on a remote client.

5.4. Building your experiment using plug-ins.

(1) experiment definition tables

Example of a common BCI experiment using a modulair approach with plug-ins.
To assist in developing your BCI experiment, a set of example files are included to demonstrate how to structure/modularize them. This setup tries to prevent usage of duplicate tables among the set of tables required for the definition of a common bci-experiment. The approach is to separate the BCI experiment into different parts or blocks, i.e., a practice, training, classifying, and feedback block. The way it is organized in these tables also shows how information from one block can easily be handed over to the next one, or vice versa how it can be retrieved from a previous one.
There can be quite a lot of overlap in information to be defined between these blocks. For example the instruction and BCI-specific markers in use. Therefore, this kind of information can better be put together in a separate table from which the other tables take their information (by referencing). Changes will then automatically migrate into all different blocks involved in your BCI experiment.

The tables involved have the following meaning:
mybci_train.xls Defines the actions for the training part or block, for every tone marker it collects data and corresponding class labels. Information is stored in variable train (Matlab type structure) and responsible code is implemented in function train.m. At startup function init_train allocates/initializes memory for train.data and train.labels or has already data available from a previous run of this train block for the same subject, the same day. This is implemented via the 'load' keyword for variable train, which means content will be loaded from disk. Content originates from a previous block that issued a 'save' keyword for the same variable. This way, information can be easily exchanged beween the different blocks or over repeated runs. The tone_count and sequence_count variables make it very easy to adapt execution to the stimulus and/or sequence number (for example separate the odd and even stimuli).Every time one of the tones marker arrives, the tone_count variable is increased by one (modifying action: $self+1), and for each of the seq_tones markers tone_count is reset to zero and sequence_count is increased by one. Furthermore, this table uses a number of referenced tables (or sheets): @MMMinit.xls; @mybci_common.xls. The @MMMinit.xls file handles installation of the acquisition hardware (Acq), adds default actions that are common to all experiment, like specific warning messages (Common), or actions to be executed in response to button presses (Buttons). The @mybci_common.xls file defines the tone, sequence marker, and actions related to incoming instruction markers.

mybci_train.xls Actions

markertimefunction hdrtone_countsequence_countclasstrainfeatures
BS_EXIT savesave
@MMMinit.xls Acq
BS_INITEVENTinit_train 00[][],load,put[],load,put
@mybci_common.xls tonesEVENT $self+1,getgetget
DATAtrain get get,putget,put
@mybci_common.xls seq_tonesEVENTreset_train get0,get$self+1,getgetget,putget
@mybci_common.xls Instruction
@MMMinit.xls Common
@MMMinit.xls Buttons

mybci_train_classifier.xls Defines the actions that are needed to train the classifier. All actions are connected to the BS_INIT marker. It loads the train data and associated class labels from disk via the 'load' keyword and executes the init_classify and cv_trainclf functions to get the classifier. The function bs_insert_marker('stop_experiment',0) tells BrainStream? to exit. During exit the actions connected to BS_EXIT will execute, which in this case is only a 'save' action to store the classifier variable to disk.

mybci_train_classifier.xls Actions

markertimefunction hdrtrainclassifier
BS_EXIT save
%NB: set data source to empty
@MMMinit.xls File
BS_INITEVENTinit_classify, cv_trainclf, bs_insert_marker('stop_experiment',0) get[],load[],get,put
@MMMinit.xls Common

mybci_feedback.xls Defines the actions for the actual feedback experiment. The structure is comparable to the train block. The classifier information is retrieved from the previous block via the 'load' keyword for variable classifier.

mybci_feedback.xls Actions

markertimefunction hdrtone_countsequence_countclassclassifierfeedback
BS_EXIT save
@MMMinit.xls Acq
BS_INITEVENTinit_feedback 00[][],load,put[],get,put
@mybci_common.xls tonesEVENT $self+1,getgetget
DATAfeedback get get,putget,put
@mybci_common.xls seq_tonesEVENTreset_feedback get0,get$self+1,getgetget,putget
@mybci_common.xls Instruction
@MMMinit.xls Common
@MMMinit.xls Buttons

mybci_common.xls tones

marker
tone1
tone2

mybci_common.xls seq_tones

markertimeclass
stone1EVENT1
stone2EVENT2

mybci_common.xls Instruction/Pictures Define which instruction picture to display for the markers involved. This is just an arbitrary implementation, which can also be achieved in many different ways.

mybci_common.xls Instruction

markertimefunctionPictureImageInstructFig
BS_INITEVENTInitTriggerInstruction({'continue.jpg', 'fixation_cross.jpg', 'end.jpg', 'final.jpg', 'instruction1.jpg', 'instruction2.jpg', 'instruction3.jpg', 'welcome.jpg'})[],get,put[],get,putInitInstructionFigure()
@PicturesEVENTTriggerInstructiongetgetget,put

mybci_common.xls Pictures

markertimePicture
fixation_crossEVENT'fixation_cross.jpg'
continueEVENT'continue.jpg'
endEVENT'end.jpg'
finalEVENT'final.jpg'
instruction1EVENT'instruction1.jpg'

mybci_common.xls DataSelection

markerbegintimeendtime
tone101
tone201

mybci_common.xls Dictionary

markertypevalue
baselinestimulus1
tone1stimulus18
tone2stimulus19
stone1stimulus20
stone2stimulus21
fixation_crossstimulus101
continuestimulus102
endstimulus103
finalstimulus104
instruction1stimulus105
button
response
129

The first three tables define the training, train_classifier and feedback blocks. The way these tables are connected will be discussed here. It is a useful feature of BrainStream? that it supports a modular definition of the different functional parts within a block and includes features that makes it easy to exchange information between the different parts of a BCI experiment.

A modular setup can be recognized by the '@' character in front of a marker name. It tells BrainStream? to look for another table, either included in the same (another sheet) or another excel file. if an extension is used for the name just behind the '@' character, it refers to another excel file and the next item specifies which sheet to use from this file. Such reference to another table can also define its own actions (also at different time points). These actions then are default actions to all markers defined in the referenced table. For example in table mybci_train.xls it includes following reference: @mybci_common.xls tones. It means all markers' actions of the table it is referring to (i.e., tone1 and tone2), will be added to this experiment. The following default actions:

marker time function hdr tone_count sequence_count class train features
@mybci_common.xls tones EVENT $self+1,get get get
DATA train get get,put get,put

will all be added to both markers tone1 and tone2. In this case there are only two markers involved, however, dependent of the specific experiment being developed it could be quite a large list. If for some reason, some of the markers of this list need different or extra actions defined, they can be overwritten/added in this referred table by changing the actions for these markers only. In many cases, a referenced table is used without the defaults and it just adds markers' actions to the experiment, then it looks like this: @MMMinit.xls Acq. No defaults are specified here and all actions are added to this experiment just as they are specified in the referenced table. There is no limit to the number of times a table is referring another table. This means table1 can refer to table2, table2 on its turn can refer to table3 and so on. This setup uses a separate file (mybci_common.xls) to define the actions or markers that are common to all or most parts of the complete BCI experiment. By referring to this table, changes that have effect on all parts of the experiment only need to be made in one table. Managing duplicate tables means a lot of extra work to be done and more importantly, it is very often a source of errors. Modularization of tables also serves the very relevant goal of encouraging sharing of tables among different people, institutes etc.

To exchange information between the different sub-parts of an experiment, user variables can easily be stored to disk using the 'save' and retrieved from disk using the 'load' keyword for corresponding variable in the experiment definition table. If the 'save' keyword is used at different time points during running a block, copies of earlier stored content are preserved in the RunFolder? . The last saved value can always be retrieved from the SessionFolder? . See ..... for more details about how BrainStream? can be instructed where to store output. A subsequent block can issue a 'load' keyword at startup/initialization to retrieve final content of the variable from a previous block. This way information can propagate (and meanwhile extended) from a first block to the last block under the presumption that users take care of proper placements of the 'save' and 'load' keywords.

5.5 Available plug-ins and/or plug-in tables

Plug-ins are pieces of code that add some well described functionality. Plug-in tables not only involve code, but also add meaningful actions connected to incoming markers.

Every user can contribute to BrainStreams? library. Put useful plug-in tables together with required source code in BrainStreams? library folder.

Independent

Bad channel detection

Bad channel detection plug-in tables ...

This is an independent plug-in that determines bad channels based on the following criteria:


Using such plug-in requires the following actions:
- define a substitute in the dictionary for the marker to which evaluate_artifacts should be merged!

marker type value substitute
mymarker stimulus 100 evaluate_artifacts, mymarker

- add @bcd.xls bcd reference in table
- in block settings define bcd client: Clients.bcd = 'local' (for parallel execution set it to another client)
- add ..brainstream/lib/bcd folder to the Matlab path!
you do not need to add gets or puts for the bcd variable, this is handled by the bcd plug-in

Dependent

Topoplot

This is a dependent plug-in that displays a topoplot of data delivered to it by the user. Using such plug-in requires the following actions:
- in your user function define content for event.topoplot.data (channels x 1)
- define a substitute in the dictionary for the marker to which update_topoplot should be merged!

marker type value substitute
mymarker stimulus 100 mymarker, update_topoplot

- add @topoplot.xls Topoplot reference in table
- in block settings define topoplot client: Clients.topoplot = 'local' (for parallel execution set it to another client)
- add ..brainstream/lib/topoplot folder to the Matlab path!
you do not need to add gets or puts for the topoplot variable, this is handled by the topoplot plug-in

The following block settings can be used to change some parameters:
[Topoplot]
Ylabel = 'Power [uV^2]'
title = 'Power of band'
%Amplitude = 1 % either specify Amplitude or Range
Range = {0 1}

Topoplotlp

This is a dependent plug-in that displays a topoplot of data delivered to it by the user at any moment. The difference with the Topoplot plugin is that its actions are not linked to the DATA timepoints of another marker using a substitute. Using such plug-in requires the following actions:
- in your user function define content for event.topoplot.data (RxC? sized cell-array for each topoplot in the layout with content: channels x 1)
AND send it directly to the client with the bs_send_user_client function.
- add @topoplotlp.xls Topoplot reference in table
- in block settings define topoplot client: Clients.topoplot = '123.456.1.1' or 'some_dns_name' (MUST execute in parallel mode)
- add ..brainstream/lib/topoplotlp folder to the Matlab path!
you do not need to add gets or puts for the topoplot variable, this is handled by the topoplot plug-in

The following block settings can be used to change some parameters:
[Topoplot]
Ylabel = 'Power [uV^2]'
title = 'Power of band'
layout = [2 3]; % 6 topoplots in a 2 x 3 layout as an example
Ylabel = {'Power [uV^2] class A', 'Power [uV^2] class B', 'Power [uV^2] class C'; 'Power [uV^2] class D', 'Power [uV^2] class E', 'Power [uV^2] class F'}
%Amplitude = 1 % either specify Amplitude or Range
Range = {[0.5 1], [-1 1], [0 1]; [0 10], [0.75 1], [0 0.1]}

Multiplot

This is a dependent plug-in that displays a multiplot of data delivered to it by the user. Using such plug-in requires the following actions:
- in your user function define content for event.multiplot.data (lines x channels x data)
- define a substitute in the dictionary for the marker to which update_multiplot should be merged!

marker type value substitute
mymarker stimulus 100 mymarker, update_multiplot

- add @multiplot.xls Multiplot reference in table
- in block settings define multiplot client: Clients.multiplot = 'local' (for parallel execution set it to another client)
- add ..brainstream/lib/multiplot folder to the Matlab path!
you do not need to add gets or puts for the multiplot variable, this is handled by the multiplot plug-in

The following block settings can be used to change some parameters:
[MultiPlot]
legend = {'eeg'} % Note: number of legends determines number of lines plotted per channel in each multiplot, the size of event.multiplot.data must match! (lines x channels x data)
Amplitude = 50;
Xunit = '[samples]'
Yunit = '[uV]'
Xlabel = 'time'
Ylabel = 'signal'

Viewdata

This is a dependent plug-in that displays a normal plot of data delivered to it by the user. Using such plug-in requires the following actions:
- it takes event.data.raw or event.data.eeg
- define a substitute in the dictionary for the marker to which update_viewdata should be merged!

marker type value substitute
mymarker stimulus 100 mymarker, update_viewdata

- add @viewdata.xls ViewData reference in table
- in block settings define viewdata client: Clients.viewdata = 'local' (for parallel execution set it to another client)
- add ..brainstream/lib/viewdata folder to the Matlab path!
you do not need to add gets or puts for the multiplot variable, this is handled by the multiplot plug-in


The following block settings can be used to change some parameters:
[Viewdata]
title = 'eeg_signal'
Amplitude = 50

Topic revision: r1 - 09 Nov 2009 - 17:44:45 - MarianneSeverens
 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback