Saturday, 15 March 2014

Is there any limit on the number of Scheduled Tasks that can be created in MarkLogic? -



Is there any limit on the number of Scheduled Tasks that can be created in MarkLogic? -

is there limit on:

the number of scheduled tasks can created in marklogic? the number of scheduled tasks can run @ same time?

note: took me around 20 minutes create 1000 schedule tasks on 16 gb ram machine in marklogic 7.

the next script used insert scheduled tasks:

class="lang-xq prettyprint-override">xquery version "1.0-ml"; import module namespace admin = "http://marklogic.com/xdmp/admin" @ "/marklogic/admin.xqy"; $i in 1 1000 allow $config := admin:get-configuration() allow $group := admin:group-get-id($config, "default") allow $new-task := admin:group-one-time-scheduled-task( "/tasks/write-log.xqy", "/", xs:datetime("2014-10-27t11:40:00"), xdmp:database("sampledb"), xdmp:database("modules"), xdmp:user("admin"), (), "normal") allow $addtask := admin:group-add-scheduled-task($config, $group, $new-task) homecoming admin:save-configuration($addtask)

please allow me know if marklogic defines limit of these. thanks!

if these one-time tasks, why not phone call xdmp:spawn directly?

scheduled tasks run on task server, @ runtime limited same factors ordinary tasks: thread count , queue size.

your code appears phone call admin:save-configuration 1000 times, doesn't surprise me it's slow. you're getting initial configuration 1000 times, sub-optimal. remember flwor look evaluates homecoming look every item in input sequence, filtered where expression.

the bottleneck creating scheduled tasks admin api itself, lot of internal validation of new configuration. that's fine when creating dozen new forests, isn't designed fast performance of hundreds of configuration changes.

assuming there's reason create scheduled tasks instead of calling xdmp:spawn, seek this:

xquery version "1.0-ml"; import module namespace admin = "http://marklogic.com/xdmp/admin" @ "/marklogic/admin.xqy"; allow $config := admin:get-configuration() allow $group := admin:group-get-id($config, "default") allow $now := current-datetime() allow $db := xdmp:database("sampledb") allow $modules := xdmp:database("modules") allow $user := xdmp:user("admin") allow $_ := $i in 1 1000 allow $new-task := admin:group-one-time-scheduled-task( "/tasks/write-log.xqy", "/", $now, $db, $modules, $user, (), 'normal') homecoming xdmp:set($config, admin:group-add-scheduled-task($config, $group, $new-task)) homecoming admin:save-configuration($config)

but since these one-time tasks, you're improve off xdmp:spawn. might using https://github.com/mblakele/taskbot

scheduled-tasks marklogic

No comments:

Post a Comment