websphere-mq-fte


Using -bs parameter (batch size, matches per task) in IBM MQ FTE transfer


I am trying to transfer files using IBM Websphere MQ FTE with parameter -bs (matches per task). I thought that using this parameter will split total files trasnfered into several tasks, according to its value. E.g. I used "-bs 10" and tried to transfer 16 files. The result should be two separate transfers, one with 10 files, second with 6 files. Am I correct? But for me the result is all 16 files in one transfer/task :/.
As stated in IBM documentation: By setting the batch size of a resource monitor (say, to 100) the resource monitor would create a new transfer for every 100 files it matched. So with 1000 .txt files you'd only get 10 new transfers being started.
I used these commands to create transfer.
fteCreateTransfer -sa AGENT1 -sm TQM.FTE -da AGENT2 -dm TQM.FTE -dd /var/MQFTE/test -de overwrite -sd delete -gt /var/IBM/WMQFTE/config/TQM.FTE/FTE_TEST.xml c:\\workspace\\FTE_tests\\IN\\*.xml
fteCreateMonitor -ma AGENT1 -mn FTE_TEST_MONITOR -md c:\\workspace\\FTE_tests -mt /var/IBM/WMQFTE/config/TQM.FTE/FTE_TEST.xml -tr match,test.run -pi 30 -pu seconds -f -c -bs 10.
Any suggestions? Thanks!
Your understanding is correct but the problem you have is because in your transfer template definition XML you are transferring source files "*.xml" which is a wildcard pattern and you are matching only on a single trigger file, "test.run".
It sounds like you want to transfer the files you match on as you don't know these in advance.
Your WebSphere MQ FTE directory resource monitor will be triggering x number of times on the poll where x is the number of files you have in the directory match your trigger pattern. These will then be batched into a number of transfer x/batchSize.
As you are only matching on one file, "test.run" the resource monitor triggers once. When the WebSphere MQ FTE agent process the single managed transfer request, you have told it to look for any files ending in .XML so it transfers all of them.
Take a look in the Information Centre for "Customizing Tasks with Variable Substitution" and use that functionality in your transfer template XML file.
Basically, you could, in your first command (fteCreateTransfer) change "c:\workspace\FTE_tests\IN\*.xml" to "c:\workspace\FTE_tests\IN\${fileName}" and then change the second command (fteCreateMonitor) to monitor in the same directory "c:\workspace\FTE_tests\IN" and match on "*.xml".
When the resource monitor triggers, it will replace ${fileName} with the name of one file for which it has triggered one and will repeat this x number of times, as it will trigger on every file ending in .xml. If there are 16 files, it will do this 16 times.
Based on the information you've given I think it will give you the behaviour you want.
Any further questions or clarifications you need, feel free to ask.
Hi Miroslav, I don't seem able to comment so I'll add a bit here to address your comment.
I'm glad to hear the changes suggested have given you the behaviour you were expecting. Indeed not knowing the exact name of the file to transfer is common but by using wildcards and certain naming conventions you can hopefully configure WebSphere MQ FTE resource monitor(s) to give the behaviour you want.
When using a trigger file of a certain name e.g., "test.run" the resource monitor is only looking for that file so will know nothing about any other files ending in .xml, for example. Hence why the batching only works on files that the resource monitor is triggering/matching on. Remember that the -bs option on the fteCreateMonitor command is for "matches per task" as defined in the Infocenter page for the command. So in your second case, the behaviour you see is expected.
I hope this helps!

Related Links

ANT FTE fte:filemove command no return code for large file
Syntax for PRESOURCE call using Protocol bridge agent as source in MQFTE
MQ-FTE options for email alerting on errors or monitoring (to be able to support the solution during Test and then Prod Support)?
Limit MQFTE file transfer to one file at a time
Using -bs parameter (batch size, matches per task) in IBM MQ FTE transfer
Space in Directory for WebSphere MQ FTE transfer
How to retry file transfer using transferId (IBM MQ fte)
How to handle FTE queued transfers
How to monitor FTE agent status
wmq-fte precall java failed
WMQ FTE in a Distributed Environment
FTE should not pick the file until unless file is ready for transfer
How to configure a fte ant script so it can perform few task before file transfer
How to implement logging in fte ant script

Categories

HOME
wso2
appx
vmware
cplex
at-command
jsrender
jxls
survey
win32gui
cross-browser
flyway4
swagger-ui
ios-charts
multiple-records
izpack
etl
correlation
workload-scheduler
systemc
http-status-code-504
samoa
alpine
opencover
microsoft-r
lcd
kannel
entitlements
rst2pdf
fifo
windowbuilder
skia
amazon-kinesis-kpl
overlap
automake
gesture
xenforo
ansible-playbook
opshub
sas-visual-analytics
optix
elmah
ibpy
android-mediaprojection
arena-simulation
cubic-spline
flickr-api
apache-fop
komodoedit
theming
directory-structure
composite-key
libusb-win32
flutterwave
or-tools
gridpane
pcf
xml-attribute
gridview-sorting
sdf
bgp
rvest
pickadate
vhd
teamcity-8.0
cannon.js
django-unittest
drawbitmap
cakephp-3.1
myo
lustre
oxwall
nstableviewcell
qpainter
winddk
phalanger
rdl
zend-route
has-many-through
poller
undefined-reference
app42
openlaszlo
intentservice
uv-mapping
gnu-smalltalk
soundtouch
excel-2003
enterprisedb
smtp-auth
itmstransporter
sublist
bitsharp
datawindow
onsubmit
online-compilation
assembly-loading
netdna-api
php-parser
floating
xfbml
getresponsestream
avatar
nerddinner
port-number
defensive-programming
grid-system
dbisam
wsdl.exe

Resources

Mobile Apps Dev
Database Users
javascript
java
csharp
php
android
MS Developer
developer works
python
ios
c
html
jquery
RDBMS discuss
Cloud Virtualization
Database Dev&Adm
javascript
java
csharp
php
python
android
jquery
ruby
ios
html
Mobile App
Mobile App
Mobile App