User:DVT/UserDocumentation/DVT Component Reference Guide/DVT Data ScriptableDP

ScriptableDP

Overview

The Scriptable Data Processor gives the user the ability to process the input data as needed and create the required output channel, by making use of a java script (JS). The component provides certain APIs which will be accessible from the JS to help in the different activities like obtaining data from input channel, creating output channels, provide regular expression support, provide buffer support for the input or output channels, etc.

Input Channel

One input channel for i/p records consisting of field(s)

Output Channel

The channel created by the script or the default channel created by the component

Properties

Property	Description	Type	Default
Arguments	The name of the Field(i.e. Column) to be enumerated. If the field is not found the component will not produce any output records	Text
Block Size	If 1, i/p data will be sent to the script's processInput function, every time a record is made available on the i/p. If more than 1, script will be passed records after Block Size number of records are received, to the script's processInputBlock function	integer	1
Java Script	The java script which will be used to configure and dictate the behavior of the component	File
Name	Displays the unique name for the component. Name can be changed through GSB	Text

Supported Types

The java code uses the types that implement the IDataField interface of the DVT SDK. Below is a list of the types which implement the interface and that are currently supported by ScriptableDP

IDataField implementation	Underlying basic type
Label	string
LongInt	long
ULong	long
UInteger	int
UShort	short
UByteArray	byte[]
FloatingPoint	double

Data Storage

The Component provides the following data storage which is maintained by the component, and which the script can access via the APIs provided.

Buffers: This can store records having the format of the input channel or output channel. This useful if user needs to collect the input records first before processing or preparing the output records before sending them out. User can obtain to a maximum of 8 such buffers. The number of entries that can be added per buffer is not limited.

Output Data Record: This is a record that is maintained by the component having the same format as the output record created. The record format will change if the output format is changed. This record is provided for user to store the data to send out on the output channel without using a buffer. Also since it has the output channel format is useful especially when the input channel format and the output channel formats are different.

APIs

The APIs are divided into the following categories

Category	Description
Input Channel	APIs for obtaining information about the input channel
Output Channel	APIs for working with internal output data record storage and sending data on the output channel
Field Manipulation	APIs for preserving the java code types (those that implement IDataField) used by DVT data processors when using in JS which just supports basic types and Objects
Buffer Storage Management	APIs for working with the buffer storage maintained by the component
Regular Expression Support	APIs for using regular expression
Misc	the following API for checking if strings are equal API obtaining arguments set in properties API to preserve basic type byte when manipulating value using JS

The APIs are referenced from the script by using ScriptableDP.function() where function is the API to be called

Input Channel APIs

API	Description
int getInputChannelFieldCount()	Returns the number of fields in the input channel. The input channel format can be known in the call to configure(IChannelDescriptor) which can be implemented in the JS side
String[] getInputChannelName()	Returns a string array of the field names
String[] getINputChannelTypes()	Returns a string array where the strings contain the data type of the corresponding fields
int ipChannelFieldIndex( name )	Returns the field index of the input channel field called name. If the field is not found will return a -1

Output Channel APIs

API	Description
createChannel( name[], type[] )	Creates the output channel for the Scriptable DP. Takes string array of field names and string array of corresponding field types
IDataField[] getAllOpChannelFields()	Returns record maintained by ScriptableDP component for storage of output data. This API could be used as a parameter to the sendOutput( IDataField data[]). For performing operations on the returned IDataField[] see section on Field Manipulation APIs
IDataField getOpChannelField( index )	Returns a field of the record maintained by ScriptableDP for storage of output data. The Field index is taken as parameter. For performing operations on the returned IDataField see section on Field Manipulation APIs
putAllOpChannelFields( IDataField[] data )	Sets the output data record maintained by the component with the array of fields passed as parameter. The array must correspond to the format of the fields of the output channel
resetopChannelFields()	Clears the contents of the output data record storage maintained by the component
setOpChannelField( name, data )	Sets the value of the field called name in the output data record maintained by the component. The Data type should match the basic types of the corresponding output field. Allowed types, long and String. Also allowed is data as IDataField (could be obtained from another API or via input channel). For IDataField the type implementations should match. Supported types float and byte[] are set using specific APIs
setOpChannelField( index, data )	Same as the previous API except that instead of taking the Field’s name takes it’s index
setOpChannelFloatField( name, data )	Used to set the value in output data record of fields which have implemented IDataField as FloatingPoint. Will set the field called name with value data
setOpChannelFloatField( index, data )	Same as previous API except that the index of the output data record field is specified instead of the name
setOpChannelBlockField( name, data[] )	Used to set the value in output data record of fields which have implemented IDataField as UByteArray. Will set the field called name with value data[].
setOpChannelBlockField( index, data[] )	Same as previous API except that the index of the output data record field is specified instead of the name
int opChannelFieldIndex( name )	Return the index number for output channel field called name. If the field is not found, returns a -1.
sendOutput( IDataField data[] )	Send the array of fields to the output channel. The format of the array of fields should match that of the output channel that has been configur
sendOutput()	Send the contents of the output data record maintained by the component on the output channel. This will not clear the contents in the output data record. User needs to call the resetopChannelFields() API if that is what is needed

Field Manipulation APIs

API	Description
long getFieldLongValue( IDataField field )	Obtain the basic type long from an IDataField object which is of type LongInt. This API can be used to obtain the basic type of the IDataField which could be returned by another API
setFieldLongValue( IDataField field, newVal )	Set a value of type long to a field of type LongInt
String getFieldStringValue( IDataField field )	Obtain the basic type long from an IDataField object which is of type LongInt. This API can be used to obtain the basic type of the IDataField which could be returned by another API.
setFieldStringValue( IDataField field, newVal )	Set value of type String to a field of type Label
byte getFieldByteValue( IDataField field )	Extracts the byte value present in a String i.e. in type Label of IDataField. Needed as JS does not have primitive type byte support and no way of converting a String to byte. This could be needed if forming data for a Block Field of type UByteArray which has primitive type byte[] for storage
byte[] getFieldBlockValue( IDataField field )	Returns a block of data in the form of bytes[] from a field of IDataField implemented as UByteArray
int getFieldBlockLength( IDataField field )	Returns the length of the block of data if the IDataField would be implemented as a UByteArray

Buffer Storage Management APIs

API	Description
int getFreeBuffer()	Returns the handle of a free buffer, the buffer is then removed form the free list. It is recommended to make this call in the reset function of the JS and not in configure (*i.e. if these functions are implemented by the JS)
int recordsInBuffer( handle )	Returns the number of records in buffer
addRecordToBuffer( handle, IDataField fields[] )	Add a record to a buffer. If this is the first record then all subsequent records should have the same format as the array of fields being passed here. If it is not the first record. Array of fields passed should have the same format as the array of fields of records already present in the buffer. These buffers can be used easily to store records having the input channel format or the output channel format
IDataField[] getRecordFromBuffer( handle, record_index )	Returns a record from the buffer as an array of IDataField. For performing operations on the returned IDatafield[] see section on Field Manipulation APIs
setRecordInBuffer( handle, record_index, IDataField data[] )	Over writes a record with the IDataField array parameter. The Field array has to match that of the format of records in the buffer
IDataField getFieldFromBufferRecord( handle, record_index, field_index )	Obtain a specific field from a specific record of a buffer. Note that if the buffer is storing records corresponding to either the input or output channel format, user could use the ipChannelFieldIndex( name ) or the opChannelFieldIndex( name ) API as the field_index parameter. For performing operations on the returned IDatafield see section on Field Manipulation APIs
setFieldInBufferRecord( handle, record_index, field_index, data )	Over write the value of a specific field in a specific row in a buffer. The parameter data should be of type long, String or IDataField
setFloatFieldInBufferRecord( handle, record_index, field_index, data )	Over write the value of a specific field in a specific row in a buffer. The parameter data should be of type float and the field being set as type FloatingPoint
setBlockFieldInBufferRecord( handle, record_index, field_index, data )	Over write the value of a specific field in a specific row in a buffer. The parameter data should be of type byte[] and the field being set as type UByteArray
clearBuffer( handle )	Deletes all the records in a buffer

Regular Expression APIs

API	Description
int regexPatternNumGroups( String pattern )	Returns the number of Groups(i.e. Bracket pairs in the pattern). E.g. “ac((b+)(\d+))” has 3 groups
int regexNumMatchOccurences( String input, String pattern )	Returns the number of times the pattern matches in the string E.g. For string “abccabcc” and pattern “ab” matches twice
boolean regexMatches( String input, String pattern )	Check to see if the pattern matches a sequence in the string at least once
boolean regexMatches( String input, String pattern, int occurrence )	Check to see if the pattern matches at least “occurrence” number of times in the string. E.g. For string “abccabcc”, pattern “ab” and occurrence 3, the API will return false as there are only two occurrences. For occurrence set as value 1 or 2 would have returned true
String regexMatchedString( String input, String pattern )	Returns the first sub string which matches the regular expression pattern. E.g. For string “abbbcccabcc” and pattern “ab+”, API will return “abbb”
String regexMatchedString( String input, String pattern, int occurence )	Returns the “occurrence”th matched substring. E.g. For string “abbbcccabcc”, pattern “ab+” and occurrence 2, API will return “ab”. If occurrence was 3 will return empty string “” as there is no third occurrence of the pattern
String regexMatchedGroup( String input, String pattern, int group_no )	Returns the substring corresponding to a particular group in the pattern for the 1st match of the pattern in the string E.g. For string “abbb12345cccab56cc” and pattern “a((b+)(\d+))”. Here group 1 is ((b+)(\d+)), group 2 is (b+) and group 3 is (\d+). If the API is called with group_no = 3, “1234” will be returned
String regexMatchedGroup( String input, String pattern, int group_no, int occurence )	Returns the substring corresponding to a particular group in the pattern for the “occurrence”th match of the pattern in the string. E.g. For string “abbb12345cccab56cc” and pattern “a((b+)(\d+))”. Here group 1 is ((b+)(\d+)), group 2 is (b+) and group 3 is (\d+). If the API is called with group_no = 3 and occurrence = 2, “56” will be returned

Misc APIs

API	Description
boolean areStringsEqual( String a, String b )	Used to compare string value. This is provided because sometimes the strings may be in objects on the JS side and therefore == will not work. This API will use the equals method in Java code to do the comparison
byte manipulateFieldByteValue( byte value )	This API is provided because JS does not support primitive type byte. It does nothing but returns the passed value as type byte. This is to be used if user wants to manipulate a byte value returned by some other API. If an operation was performed on this JS would convert the result to an Object and that would cause an error if the result was being passed down to an API expecting parameter byte. E.g. if a variable x contains a value of type byte which was returned form some API and now user wanted to subtract 10 form the byte. User would have to do the following x = manipulateFieldByteValue( x – 10) and not x = x – 10
String getArguments()	Returns the string which has been set for the “Argument” property of the Scriptable DP in the property window

Accesing DVT Data Model

The DVT data model can be accessed via ScriptableDP.model. From this user gets access to all the rest of the components in the solution and can even change their properties etc.

DVT Data Processot Java Code methods that JS can implement

Method	Interface	Default java code functionality if JS does not implement	Java code functionality even if JS implements
configure	IDataProcessor.IInput	Configures output channel to the same format as input channel
processInput	IDataProcessor.IInput	Send data received as is to output channel
processDiscontinuity	IDataProcessor.IInput	Does Nothing	Send discontinuity message to output
reset	IDataProcessor	Any initialization needed at reset for component	Mark all 8 internal Buffers storage as free

Example Solutions

<install>\dvt\Examples\ScriptableDP\Block\Block.sol

<install>\dvt\Examples\ScriptableDP\BufferManagement\buffermgt.sol

<install>\dvt\Examples\ScriptableDP\Control\control.sol

<install>\dvt\Examples\ScriptableDP\OutputChannel\opchannel.sol

<install>\dvt\Examples\ScriptableDP\parameters\parameters.sol

<install>\dvt\Examples\ScriptableDP\reconfigure\reconfigure.sol

<install>\dvt\Examples\ScriptableDP\RegEx\regexSDP.sol

Example	Description
Block	Demonstrates how to deal with a block of data of type UByteArray
buffermgt	Demonstrates use of the buffer management APIs
control	Accesses Data model to change the properties of some other components in the Solution
opchannel	Demonstrates use of the output channel APIs
parameters	Shows how to use parameters passed from the components property window
reconfigure	Shows how to reconfigure the output channel based on changing input
regexSDP	Uses all the regular expression APIs supported

Also See

TODO: Add references to tutorials in which component is used

Enhancement Requests

Gforge Tracker ID: TODO

Known Issues

Gforge Tracker ID: