websphere-mq-fte


Using -bs parameter (batch size, matches per task) in IBM MQ FTE transfer


I am trying to transfer files using IBM Websphere MQ FTE with parameter -bs (matches per task). I thought that using this parameter will split total files trasnfered into several tasks, according to its value. E.g. I used "-bs 10" and tried to transfer 16 files. The result should be two separate transfers, one with 10 files, second with 6 files. Am I correct? But for me the result is all 16 files in one transfer/task :/.
As stated in IBM documentation: By setting the batch size of a resource monitor (say, to 100) the resource monitor would create a new transfer for every 100 files it matched. So with 1000 .txt files you'd only get 10 new transfers being started.
I used these commands to create transfer.
fteCreateTransfer -sa AGENT1 -sm TQM.FTE -da AGENT2 -dm TQM.FTE -dd /var/MQFTE/test -de overwrite -sd delete -gt /var/IBM/WMQFTE/config/TQM.FTE/FTE_TEST.xml c:\\workspace\\FTE_tests\\IN\\*.xml
fteCreateMonitor -ma AGENT1 -mn FTE_TEST_MONITOR -md c:\\workspace\\FTE_tests -mt /var/IBM/WMQFTE/config/TQM.FTE/FTE_TEST.xml -tr match,test.run -pi 30 -pu seconds -f -c -bs 10.
Any suggestions? Thanks!
Your understanding is correct but the problem you have is because in your transfer template definition XML you are transferring source files "*.xml" which is a wildcard pattern and you are matching only on a single trigger file, "test.run".
It sounds like you want to transfer the files you match on as you don't know these in advance.
Your WebSphere MQ FTE directory resource monitor will be triggering x number of times on the poll where x is the number of files you have in the directory match your trigger pattern. These will then be batched into a number of transfer x/batchSize.
As you are only matching on one file, "test.run" the resource monitor triggers once. When the WebSphere MQ FTE agent process the single managed transfer request, you have told it to look for any files ending in .XML so it transfers all of them.
Take a look in the Information Centre for "Customizing Tasks with Variable Substitution" and use that functionality in your transfer template XML file.
Basically, you could, in your first command (fteCreateTransfer) change "c:\workspace\FTE_tests\IN\*.xml" to "c:\workspace\FTE_tests\IN\${fileName}" and then change the second command (fteCreateMonitor) to monitor in the same directory "c:\workspace\FTE_tests\IN" and match on "*.xml".
When the resource monitor triggers, it will replace ${fileName} with the name of one file for which it has triggered one and will repeat this x number of times, as it will trigger on every file ending in .xml. If there are 16 files, it will do this 16 times.
Based on the information you've given I think it will give you the behaviour you want.
Any further questions or clarifications you need, feel free to ask.
Hi Miroslav, I don't seem able to comment so I'll add a bit here to address your comment.
I'm glad to hear the changes suggested have given you the behaviour you were expecting. Indeed not knowing the exact name of the file to transfer is common but by using wildcards and certain naming conventions you can hopefully configure WebSphere MQ FTE resource monitor(s) to give the behaviour you want.
When using a trigger file of a certain name e.g., "test.run" the resource monitor is only looking for that file so will know nothing about any other files ending in .xml, for example. Hence why the batching only works on files that the resource monitor is triggering/matching on. Remember that the -bs option on the fteCreateMonitor command is for "matches per task" as defined in the Infocenter page for the command. So in your second case, the behaviour you see is expected.
I hope this helps!

Related Links

ANT FTE fte:filemove command no return code for large file
Syntax for PRESOURCE call using Protocol bridge agent as source in MQFTE
MQ-FTE options for email alerting on errors or monitoring (to be able to support the solution during Test and then Prod Support)?
Limit MQFTE file transfer to one file at a time
Using -bs parameter (batch size, matches per task) in IBM MQ FTE transfer
Space in Directory for WebSphere MQ FTE transfer
How to retry file transfer using transferId (IBM MQ fte)
How to handle FTE queued transfers
How to monitor FTE agent status
wmq-fte precall java failed
WMQ FTE in a Distributed Environment
FTE should not pick the file until unless file is ready for transfer
How to configure a fte ant script so it can perform few task before file transfer
How to implement logging in fte ant script

Categories

HOME
vim
cloud
c#-4.0
plot
lodash
gis
programming-languages
icloud
azure-media-services
cross-validation
python-unittest
reveal.js
u-sql
apache-cayenne
decimal
django-simple-history
workload-scheduler
windows-7-x64
solaris-10
karma-jasmine
scichart
dbext
ml
opencover
pepper
cas
facebook-access-token
traffic
fluentvalidation
jspm
xquery-3.0
oracle-fusion-middleware
winrt-xaml-toolkit
lxd
azure-sql-database
language-concepts
bower-install
service-discovery
vao
taffy
boost-multi-index
nstextview
qtwebview
akka-cluster
gcsfuse
redundancy
quartz-composer
slicknav
file-writing
bstr
lua-5.1
whois
ptrace
pyke
netmq
freedesktop.org
hsv
prerequisites
lemon
google-places
marching-cubes
python-3.2
django-unittest
pundit
dereference
dia
id3v2
gulp-less
neolane
googlemock
code-access-security
castle
sgen
funcunit
p4java
kgdb
dealloc
centos5
jsctypes
aqtime
lcs
smtp-auth
referrer
unc
google-email-migration
pysimplesoap
twrequest
parametric-equations
buildr
eclipse-templates
deobfuscation
subgurim-maps
horizontal-accordion
vc90
yagni
pascal-fc
paul-graham

Resources

Mobile Apps Dev
Database Users
javascript
java
csharp
php
android
MS Developer
developer works
python
ios
c
html
jquery
RDBMS discuss
Cloud Virtualization
Database Dev&Adm
javascript
java
csharp
php
python
android
jquery
ruby
ios
html
Mobile App
Mobile App
Mobile App