Concurrent Queries

I’d like to understand omniscidb query concurrency. Are queries run in parallel? Is there a way to control the number of executors/workers?


Hi @Ray1,

Historically we have focused on intra-query parallelism; that is, executing one query at a time, but each query step in parallel. This is a natural fit for the massive SIMD parallelism you get from GPUs (though note that we execute query steps in parallel on CPU as well). We can scale across multiple GPUs and multiple servers as well, allowing us to scale out to bigger and bigger datasets.

However, we have recently invested in inter-query parallelism; that is, executing stages of a query in parallel, and executing multiple queries across multiple devices in parallel as well. These features allow us to smooth out the load of multiple queries from multiple users across available execution devices. For example, a SQL query with a HAVING clause will execute in two steps – the first step executes the SQL prior to HAVING; the second step executes the HAVING as a filter on top of the prior steps result. The HAVING step is typically over smaller data size than the inputs, thus while we are executing the HAVING clause, we can make other GPU or CPU threads available to other queries, either from the same dashboard or from other users

The initial work for this functionality has already to landed and can be enabled using the num-executors flag in the omnisci.conf file or on the command line. Note that this functionality is still in “beta”, and we are making improvements and refinements on a weekly basis. At this time we recommend a num-executors value of 1/2 to 3/4 the number of available GPUs (our typical customer deployments are 8 GPUs, and we run with either 4 or 6 executors there). Concurrent query execution across multiple GPUs is not yet available, but improvements in query pipelining and concurrent execution of other stages (e.g. reducing the results from multiple GPUs to return a single result set) is in place. You can try this functionality in either our open source version or our enterprise edition trial


Thanks Alex, this very helpful.